Troubleshooting startup issues with managed application having Authenticode signature

Authenticode , also referred as a digital signature , is an industry standard of verifying the source of an application by setting up a trust relationship with the publisher of the application. However, digitally signing your managed application might cost you in terms of application startup time.

Following are the facts:

1. When the CLR loads an assembly which has an Authenticode signature, it will always try to verify the signature by connecting to the Internet to generate Publisher evidence for the assembly. This is true even if there is no internet connectivity available.

2. In this verification, the CLR tries to search all the certificates that are no longer valid and are revoked by the Certificate Authority (CA). These certificates are part of a special list called the Certificate Revocation List (CRL). The verification process might take a long time, since it requires contacting the CA online to download the latest CRL. This delay also depends on the network configuration.

The delay is more visible for a managed service which might be time critical; eventually the service may not start on time. By default, Code Access Security does not check for Publisher evidence. Unless your computer has a custom code group based on the PublisherMembershipCondition class. In most cases, this unnecessarily exhausts the start up cost, so disabling the signature verification is a better choice. When you disable signature verification, the .NET runtime no longer receives publisher evidence.

To start troubleshooting:

1. Install Debugging Tools for Windows (You can optionally use System Internals, Process Explorer to analyze the callstack, but WinDbg is my choice)

2. Use ADPlus to collect a hang dump for the application. Remember the hang dump should be taken at the point when application appears to be in a hang state. Also, it is better to take multiple hang dumps with an interval of 3 to 4 seconds just to ensure that we capture the correct callstack.

3. Open the hang dumps in WinDbg and start analyzing the callstack. However, to view correct callstack, do not forget to specify the public symbol server path. You can then add a local cache to the symbols path. The path should finally look similar to

“srv*C:\Public Symbols*msdl.microsoft.com/download/symbols

4. Now run, WinDbg command “kbn”, to see the callstack with function arguments. You might see a similar callstack shown bellow with CryptRetrieveObjectByUrlWithTimeout API. This function clearly indicates that the application has a timeout as a part of the certificate revocation process.

0:000> kbn

 # ChildEBP RetAddr Args to Child

00 0012d980 77bb9254 770cc244 00000510 00000000 ntdll!KiFastSystemCallRet

01 0012d984 770cc244 00000510 00000000 0012d9cc ntdll!ZwWaitForSingleObject+0xc

02 0012d9f4 71066247 00000510 00003a98 00000000 kernel32!WaitForSingleObjectEx+0xbe

03 0012da28 71061f7c 0034ac84 00000002 00202005 cryptnet!CryptRetrieveObjectByUrlWithTimeout+0x1a5

04 0012db18 710627a1 0034ac84 00000002 00202005 cryptnet!CryptRetrieveObjectByUrlW+0xcc

05 0012dbb8 710691be 00000000 0034ac84 00000002 cryptnet!RetrieveObjectByUrlValidForSubject+0xa0

06 0012dc18 71062556 00000000 00000000 00000000 cryptnet!RetrieveTimeValidObjectByUrl+0x15c

07 0012dcc4 71063fc1 0000004e 0034ac78 00000000 cryptnet!CTVOAgent::GetTimeValidObjectByUrl+0x178

08 0012ddac 71063d98 00000003 00348a20 00000002 cryptnet!CTVOAgent::GetTimeValidObject+0x4a8

09 0012dddc 71063428 00000003 00348a20 00326af0 cryptnet!FreshestCrlFromCrlGetTimeValidObject+0x2d

0a 0012de20 71069283 00000003 00348a20 00326af0 cryptnet!CryptGetTimeValidObject+0x58

0b 0012de9c 71063b1c 00000003 00348a20 00348a20 cryptnet!GetTimeValidCrl+0x2cb

0c 0012dee0 71063992 00348a20 00326af0 0012df18 cryptnet!GetBaseCrl+0x34

0d 0012df8c 75ce77d5 00000001 00000001 00000001 cryptnet!MicrosoftCertDllVerifyRevocation+0x163

0e 0012e014 75ce7641 00000001 00000001 00000001 crypt32!I_CryptRemainingMilliseconds+0x2aa

0f 0012e098 75ce7a26 00000001 00000001 00000001 crypt32!CertVerifyRevocation+0xd4

10 0012e144 75ce7838 002fa8b8 00000000 00000000 crypt32!CChainPathObject::CalculateRevocationStatus+0x2d0

11 0012e188 75cf6c44 002fa8b8 0033e230 0033de70 crypt32!CChainPathObject::CalculateAdditionalStatus+0x152

12 0012e250 75cf69a5 002fa8b8 00348980 0033de70 crypt32!CCertChainEngine::CreateChainContextFromPathGraph+0x23e

13 0012e288 75cfbe14 00348980 00349b7c 0033de70 crypt32!CCertChainEngine::GetChainContext+0x46

Going further, you can verify the exact URL being searched for the revocation by switching to frame 3 and looking at the first argument “0034ac84 . You may even find the timeout value looking at the fourth argument “00003a98”.

To find this, run dump raw stack command, “dds” on to the base ChildEBP pointer “0012da28”.

0:000> .frame 3

03 0012da28 71061f7c cryptnet!CryptRetrieveObjectByUrlWithTimeout+0x1a5

0:000> dds 0012da28

0012da28 0012db18

0012da2c 71061f7c cryptnet!CryptRetrieveObjectByUrlW+0xcc

0012da30 0034ac84à URL being searched for the revocation

0012da34 00000002

0012da38 00202005

0012da3c 00003a98à timeout value

 

0:000> du 0034ac84

0034ac84 "crl.verisign.com/pca3.crl"

0034acc4 ""

 

0:000> ? 00003a98

Evaluate expression: 15000 = 00003a98

After confirming it to be a certificate revocation issue, you might start with fixing the issue. If your managed application is running on Windows XP with .NET Framework 2.0, you would need to install the hotfix support.microsoft.com/kb/936707. This hotfix adds the “generatePublisherEvidence” configuration setting to the .NET Framework. After you apply this hotfix, you can then use a configuration file to disable signature verification. Add the following code to the <ApplicationName>.exe.config file for the .NET Framework 2.0 managed application.

<configuration>

<runtime>

<generatePublisherEvidence enabled="false"/>

</runtime>

</configuration>

Note: The hotfix is only required for .NET 2.0 Framework on Windows XP.

 

This time, the application should start faster. For redistribution purposes, I would recommend you to use the .NET Framework 2.0 Service Pack 1 for Windows XP and Microsoft .NET Framework 3.5 for Windows Vista and Longhorn.

 

Now, assume you happily created the configuration file to disable the verification and you decide to create a setup deployment. Going further to improve your application startup time, you may decide to NGen the managed assemblies while installation. Be careful, because soon you will fall in the same trap of startup performance, and this time it would be your setup taking more time. This is because NGen will still try to look for the CRL even if you have a valid configuration file. You might think now, passing the config file to NGen.exe using /Execonfig option will resolve the issue. NGen would completely read the configuration file, but will ignore the “generatePublisherEvidence”. This is because, “generatePublisherEvidence”config switch kicks in only at runtime - not at NGen time.

Workarounds that can be used:

1. Do not use Authenticode signatures for your managed assemblies. Use strong-named signing instead.

2. Schedule NGen in the background so that the hanging Authenticode signature verification does not block setup. This can be done using “ngen.exe install <ApplicationName>.exe /queue”.

 

If you are within a reliable environment, you may disable the certificate revocation lookup for your entire system, by turning off the “Software Restriction Policies” (technet.microsoft.com/en-us/library/bb457006.aspx#EAAA). This can be done by unchecking an option from Internet Explorer, “Tools | Options |Advanced | Security |Check for publisher’s certificate revocation”.

 

- Gaurav Patole.

 

Developer Support VC++ and C#