MSI - Generate SHA2 certificate with OpenSSL and sign a MSI installer project, use 'Application Control Policies'-'AppLocker' to block unsigned code

Some time ago I had to troubleshoot an issue with signed native C++ XLL add-ins for Excel, which didn't get
loaded with certain combinations of Excel and Windows.

Symptom
 - A client has developed an unmanaged XLL add-in for Excel, which has been signed with a code signing certificate that uses SHA2 as hashing algorithm.- Everything works as expected when the XLL is deployed to Windows XP + Excel 2007 machines.- However, when loaded in Windows 7 + Excel 2007 environments, Office will flag the add-in signature’s as invalid and it refuses to load this extension. The  security settings are set to allow only signed add-ins to load. - On the other hand, the same XLL works fine in Excel 2010 and 2013. - This issue seems to reproduce even when using simple workbooks which contain macros, if these files get signed with paid certificates, as well as with free versions (as long as they use a SHA2 encryption) obtained from OpenSSL.com (https://www.openssl.org/docs/HOWTO/certificates.txt). 
Cause
 - Our developers found that it was a flaw in Office 2007, where an internal buffer length issue causes the error in Windows 7. - Office 2010 had been affected by this problem as well, but it got addressed in a hotfix: https://support.microsoft.com/kb/2598139. Unfortunately since Office 2007 is out of mainstream support, its code remained unpatched. 

 
Initially I tried to narrow the problem down, as we needed to understand if the Office client or Windows OS was
responsible for the aforementioned behavior.

The simplest test would have been to generate a SHA2 certificate and check if Windows allows a signed
executable (it can be anything as long as it has no connection with Office) to run.

For the executable, I decided to use a basic MSI Installer project and for the SHA2 signature validation I used a
built-in security feature named 'AppLocker', which extends the 'Software Restriction Policies'.

Because I couldn't find a very clear set of instructions on how one can generate its own certificates with OpenSSL
and apply AppLocker restrictions for MSI projects, I decided to share this info with you. 

Generate a self-signed Root CA, an Intermediate CA and a SHA2 certificate 

1.     Download the compiled OpenSSL binaries from this page: https://www.openssl.org/related/binaries.html;
2.     Install Microsoft Visual C++ 2008 Redistributable Package (x86) ;
3.     Extract the OpenSSL archive and run the installer project (for my test I used this path: C:\OpenSSL-Win32);
4.     Create another folder to keep track on the output files you are going to generate (C:\OpenSSL);
5.     Open a CMD window and change directory to the location where OpenSSL.exe is found
       (C:\OpenSSL-Win32\bin);
6.     Run this command create a root CA key file:

openssl genrsa -out "C:\OpenSSL\Test_CertAuthority.key" 4096

 

7.     Now run this command to generate a root CA certificate based on the key file created at the previous
       step; You will be prompted to enter some details;

 Note:  OpenSSL.exe will complain that it can't find a required .cfg file; Use an additional -config parameter
            to provide a valid path;

openssl req -new -x509 -days 365 -key "C:\OpenSSL\Test_CertAuthority.key" -out "C:\OpenSSL\Test_CertAuthority.crt" -config "C:\OpenSSL-Win32\bin\openssl.cfg"

... fill-in this info. when prompted:

Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:.Locality Name (eg, city) []:.Organization Name (eg, company) [Internet Widgits Pty Ltd]:TestMSOrganizational Unit Name (eg, section) []:TestOrgUnitCommon Name (e.g. server FQDN or YOUR name) []:CristiBEmail Address []:cristib@microsoft.com

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:.An optional company name []:.

 Note:  I could have linked my certificate directly to the Root CA, but it would be safer to keep it isolated and
           introduce an intermediate CA layer in between .. for more details, please read these articles:
           >  Certification Authority Trust Model (link
           >  Benefits of Multiple-Level Certification Hierarchies (link);

8.     Create an IA key and a CSR file: 

openssl genrsa -out "C:\OpenSSL\Test_IntermediateCertAuthority.key" 4096openssl req -new -key "C:\OpenSSL\Test_IntermediateCertAuthority.key" -out "C:\OpenSSL\Test_IntermediateCertAuthority.csr" -config "C:\OpenSSL-Win32\bin\openssl.cfg"

 .. and fill-in this info. when prompted:

Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:.Locality Name (eg, city) []:.Organization Name (eg, company) [Internet Widgits Pty Ltd]:TestMSOrganizational Unit Name (eg, section) []:TestOrgUnit_IntermediateAuthorityCommon Name (e.g. server FQDN or YOUR name) []:CristiB_IntermediateAuthorityEmail Address []:cristib@microsoft.com

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:.An optional company name []:.

 

9.     Generate an IA certificate based on the root CA and the CSR file: 

openssl x509 -req -days 365 -in "C:\OpenSSL\Test_IntermediateCertAuthority.csr" -CA "C:\OpenSSL\Test_CertAuthority.crt" -CAkey "C:\OpenSSL\Test_CertAuthority.key" -set_serial 01 -sha256 -out "C:\OpenSSL\Test_IntermediateCertAuthority.crt"

 

 

10.    Now that we have the intermediate CA key and certificate we can package them inside a PKCS12 type of
        file:

openssl pkcs12 -export -out "C:\OpenSSL\SHA2_IA_Cert.p12" -inkey "C:\OpenSSL\Test_IntermediateCertAuthority.key" -in "C:\OpenSSL\Test_IntermediateCertAuthority.cer" -chain -CAfile "C:\OpenSSL\Test_CertAuthority.crt" -passout pass:123456 -name My_SHA2_Cert

 

 

 
Before you can use these certificates, you need to import them (Start > Run > certmgr.msc):
> the top-level CA file (Test_CertAuthority.crt) goes into Trusted Root Certification Authorities;
> the intermediate CA (Test_IntermediateCertAuthority.crt) gets imported into the Intermediate
Certification Authorities
;

 

  

 
Build a very simple MSI Installer project using Visual Studio 2010 and sign its output

1.      Open a new Visual Studio Command Prompt and change directory to the 'Debug' output folder where
        the MSI project was compiled;
2.      Use the 'SignTool' utility (https://msdn.microsoft.com/en-us/library/8s9b9yaz(v=vs.110).aspx - SignTool.exe),
        which ships with Visual Studio to sign the 'Setup.exe' and ' .MSI' file;

signtool sign /v /t https://timestamp.verisign.com/scripts/timestamp.dll /f "C:\OpenSSL\SHA2_IA_Cert.p12" /p 123456 MSI_Signed_Installer.msi 

signtool sign /v /t https://timestamp.verisign.com/scripts/timestamp.dll /f "C:\OpenSSL\SHA2_IA_Cert.p12" /p 123456 Setup.exe

 

3.      Validate the signature by right-clicking on the files in Windows Explorer > Properties > Digital
Signatures
...

 
... or by using the SignTool utility:

signtool verify /v /a setup.exe

 

 

Customize AppLocker to block all other installers except those signed with our certificate

1.      Copy the output files form the MSI installer project to a machine;
2.      Copy the root CA (Test_CertAuthority.crt) and the intermediate CA
        (Test_IntermediateCertAuthority.crt) certificates in the same environment (make sure you do not
        accidentally copy the key files, which should be safely stored in a secure environment);

Note:  It is very important to deploy your certificates in the correct place. Double-clicking on them will           get them installed into your Local User's store which is not going to be seen by AppLocker.           If you're not careful, you might spend hours troubleshooting Rules that would otherwise work perfectly fine.             For AppLocker to work properly and let you block .MSI installers based on Publishers, you will            have to deploy any certificate referenced in your Rules into the Local Computer store.

Note:  For the following steps you will need to be logged on using an Administrative account.

Note:  AppLocker only applies to normal user accounts, so members of the Administrators group will not be blocked from running any MSI installers!

3.      Deploy your certificates in the Local Computer certificate store: Open a new MMC console by going
        to Start > Run > MMC.msc > File > Add/Remove Snap-in... , find the Certificates entry in the snap-ins
        list > click Add > select 'Computer account';

4.      Deploy the root CA into the Trusted Root Certification Authorities and the intermediate CA into 
        the Intermediate Certification Authorities;  

5.      Open a new MMC console > Start > Run > MMC.msc > File > Add/Remove Snap-in... and add the
        Local Computer Policy management console as shown below:

6.      Inside the Local Computer Policy management console, expand these nodes:
        > 'Local Computer Policy
           > 'Windows Settings
              > 'Security Settings
                 > 'Application Control Policies
                    > 'AppLocker
                       > 'Windows Installer Rules'
7.      Select 'Create New Rule.. ' from the right-hand side Actions panel;
8.      Apply the following settings:
       > 'Action':              Allow;
       > 'User or group': Everyone; 
       > 'Conditions':      Publisher;

       > click 'Browse' and select the signed .MSI file;

 

Note:  In case you haven't installed the root CA and intermediate CA, the AppLocker Wizard will            display the following error when you will attempt to browse for a .MSI file: The publisher information cannot be extracted from the specified file <path>. Reason: The object identifier does not represent a valid object. (Exception from HRESULT: 0x800710D8);

9.      Customize the Publisher rule, to allow only our certificate to run. You do this by dragging the slider up
        to 'Publisher';

 
10.     Type a description in the Name field to complete the Wizard.  

11.     In case AppLocker wasn't used until now, you will be prompted to create a few default rules. Go ahead
         and click YES.

12.     It seems that AppLocker will not enable your rules unless you explicitly enforce them. Inside the Local
Computer Policy
management console, expand these nodes:
         > 'Local Computer Policy
           > 'Windows Settings
              > 'Security Settings
                 > 'Application Control Policies
                    > select the 'AppLocker' node;
                       > click on 'Configure rule enforcement';
                          > mark the Windows Installer: Configured check-box;
                             > select Enforce rules and click OK;

13.     Turn on the 'Application Identity' service (Start > Run > type SERVICES.msc and hit ENTER); Find
         the aforementioned item and start it:

14.     Open CMD prompt and run 'gpupdate /force';

Test that everything works

1.       Switch to a non-privileged user account on that machine;
2.       Check your user's certificates to verify that the root and intermediate CA are present; Notice that the
         certificates inherited from the Local Machine store are marked with Status R (read-only);

3.       Check that the Setup.exe is signed and its certificate is valid;
4.       Execute the installer;

5.       Try to execute an unsigned installer package (notice that the Digital Signatures tab is missing in the
         Properties window); You should get this error:

The system administrator has set policies to prevent this installation.

 

 
6.       Also, the error gets logged in Event Viewer:

 

The installation of <MSI package> is not permitted by software restrictionpolicy. The Windows Installer only allows installation of unrestricteditems. The authorization level returned by software restriction policy was0x0.

I would have liked to mention some interesting topics related to how Certificate Authorities and AppLocker
rules work, but my article is already too long to bore you with additional stuff. But if you want to learn more, 
you may find these links useful:

Certification Authority Guidance https://technet.microsoft.com/en-us/library/hh831574.aspx;
Understanding AppLocker Rules https://technet.microsoft.com/en-us/library/dd759068.aspx;

Understanding AppLocker Rules (LINK ) -------------------------------------------------------------------

[...]

AppLocker rule behavior

If no AppLocker rules for a specific rule collection exist, all files with that file format are allowed to run. However, when an AppLocker rule for a specific rule collection is created, only the files explicitly allowed in a rule are permitted to run. For example, if you create an executable rule that allows .exe files in %SystemDrive%\FilePath to run, only executable files located in that path are allowed to run.

A rule can be configured to use either an allow or deny action:

  • Allow. You can specify which files are allowed to run in your environment and for which users or groups of users. You can also configure exceptions to identify files that are excluded from the rule.
  • Deny. You can specify which files are not allowed to run in your environment and for which users or groups of users. You can also configure exceptions to identify files that are excluded from the rule.

 [...]

 

Thank you for reading my article! If you have liked it, please use the rating
button.

I can't always manage to reply to your comments as fast as I'd like. Just drop me
an email at cristib-at-microsoft-dot-com, should you need help with understanding
or getting something in my blog to work. 

Bye :-)