The access control list is not canonical

Developing a program that does not require administrative privileges requires quite a bit of work up-front during the install. You need to make sure you do things like write registry keys to HKLM, create file system entries, and most importantly ensure that your ACLs (Access Control List) are correct to ensure smooth operation of your program while running under that non-admin account. However, sometimes this last step fails with a rather cryptic error, mentioning something along the lines of "The Access Control List is not canonical". After searching the web for solutions to this error, I couldn't find a definitive answer. One thing I did notice is if you open the 'security' dialog of the object that exhibits this issue, the Windows UI will actually recognize that there is an issue and ask if you would like to correct it. So obviously there is a way to fix this problem, but for my scenario requiring user intervention was just not good enough .. so I returned to searching. I finally came across an algorithm, buried in the depths of MSDN, which described how to fix the ordering of a DACL (Discretionary Access Control List). There was only one problem: it was written in VB. So, after a lot of hard work converting the code (just kidding, it was quite simple to convert), I produced the following helper method to attempt fixing the order of a DACL.

    internal static void CanonicalizeDacl(NativeObjectSecurity objectSecurity)
   {
       if (objectSecurity == null)
       {
           throw new ArgumentNullException("objectSecurity");
       }

       if (objectSecurity.AreAccessRulesCanonical)
       {
           return;
       }

       // A canonical ACL must have ACES sorted according to the following order:
        //   1. Access-denied on the object
        //   2. Access-denied on a child or property
        //   3. Access-allowed on the object
        //   4. Access-allowed on a child or property
        //   5. All inherited ACEs 
        RawSecurityDescriptor descriptor = new RawSecurityDescriptor(objectSecurity.GetSecurityDescriptorSddlForm(AccessControlSections.Access));

       List<CommonAce> implicitDenyDacl = new List<CommonAce>();
       List<CommonAce> implicitDenyObjectDacl = new List<CommonAce>();
       List<CommonAce> inheritedDacl = new List<CommonAce>();
       List<CommonAce> implicitAllowDacl = new List<CommonAce>();
       List<CommonAce> implicitAllowObjectDacl = new List<CommonAce>();

       foreach (CommonAce ace in descriptor.DiscretionaryAcl)
       {
           if ((ace.AceFlags & AceFlags.Inherited) == AceFlags.Inherited)
           {
               inheritedDacl.Add(ace);
           }
           else
            {
               switch (ace.AceType)
               {
                   case AceType.AccessAllowed:
                       implicitAllowDacl.Add(ace);
                       break;

                   case AceType.AccessDenied:
                       implicitDenyDacl.Add(ace);
                       break;

                   case AceType.AccessAllowedObject:
                       implicitAllowObjectDacl.Add(ace);
                       break;

                   case AceType.AccessDeniedObject:
                       implicitDenyObjectDacl.Add(ace);
                       break;
               }
           }
       }

       Int32 aceIndex = 0;
       RawAcl newDacl = new RawAcl(descriptor.DiscretionaryAcl.Revision, descriptor.DiscretionaryAcl.Count);
       implicitDenyDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       implicitDenyObjectDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       implicitAllowDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       implicitAllowObjectDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       inheritedDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));

       if (aceIndex != descriptor.DiscretionaryAcl.Count)
       {
           System.Diagnostics.Debug.Fail("The DACL cannot be canonicalized since it would potentially result in a loss of information");
           return;
       }

       descriptor.DiscretionaryAcl = newDacl;
       objectSecurity.SetSecurityDescriptorSddlForm(descriptor.GetSddlForm(AccessControlSections.Access));
   }

I hope this code saves someone out there a lot of trouble and time. Until next time …