A peek into a performance issue of a .net assembely

Performance issues are are quite tricky and might require effort to debug , albeit we have many coding practices in place to avoid such issues it might turn out to be a completely different case which may or may not be in our control. We were recently approached by one of our partners regarding a performance issue during start up on one of their .net 2.0 application which had a bunch of exes,libraries internally built as well as third party specifically on Windows Server 2008 R2 as the same application(one of the exes) started quickly in different version of Windows Server 2008 . What started as a managed performance issue reached up to a point of security . Here is how it shaped.

One of the very first things that I verify for a peformance issue is that if this had any abnormal behaviors wrt CPU usage and memory usage. As there were none, we also saw that the application was tragetting x86 architecture. There are many tools which help you narrow down performance issues, but one such tool is perfview which provides you great level of granularity . Perfview will track down the issue to event level and function level statistics. You can download perfview from www.microsoft.com/en-us/download/details.aspx?id=28567. Our client had run some other traces previously which provided clues that the issue was because of Just in time compilation. In that case Perfview provides jit stats which will clearly tell us if there is a distinction in the 2 runs.With out further ado , I collected the data for both the machines with the below options.

Once we had the data, thanks to perfview it provides a great view of the statistics per important parameters . I was interested in the jit stats of the logs . I wanted to specifically compare the 2 logs and see if there is a difference. Interestingly I did see some amount of difference. The next sensible thing to do was to create native images for all those assemblies which are listed in the below table using ngen. If you observe the below table clearly lists the assemblies where considerable amount of time is spent. The second table shows us the functions where we see lot of time is spent. To know more about how ngen improves the performance , please refer Msdn article .

Despite creating native images the performance improvement was not that great. Our client informed us that the application is still starting up slowly. I further went back to check perfview logs to see if there are any red herring. To my surprise I observed that there was one particular event in the events category it had a long delay in the “Microsoft-Windows-DotNETRuntime/AuthenticodeVerification/Start” section. It looked like it took over 20 seconds . When I tried to learn more about it, I got to know that this has more to do with Authenticode Signing . There was one blog which provides some insight into the issue and how to avoid delays due to this event. I just wanted to take a note here that our client did try the work around in their test environment and the results were quite satisfactory indeed. While this seems pretty much a valid step to take but there are some points that we will have to consider before going on the work around.

The work around suggested in there is to add an entry to config file of the exe ,

<configuration>               
<runtime>                
<generatePublisherEvidence enabled=”false” />
</runtime></configuration>

This relates to authenticode signing . What is authenticode signing?
Authenticode allows you to verify who the author of the assembly is. The only association between identity and public key created by a strong name is via word of mouth and personal trust decisions. For instance, since Microsoft ships a security policy which indicates that the Microsoft public key should be FullTrust, I can infer that Microsoft owns that key, and thus any assembly signed by the corresponding private key came from Microsoft. Authenticode on the other hand, assigns a definite identity to a key, and allows for that identity to be verified by chaining the certificate up to a trusted root. It also provides an expiration date for certificates, and allows for a compromised key to be revoked.

What happens if we add the entry to config file ?
In simple words , this authenticode is like a key to a room . We can enter the room by just disabling the lock system. So if we bypass this we might very well be able to run our application however at the cost of security . When we say security there is a caveat to it . In the blog above it tells us that if your assembly is already running with full trust permission then you can bypass this . How ever we should verify if our exe is running with fulltrust, I have a small exercise that you can run to know the trust at the end. When the CLR loads an assembly which has an Authenticode signature, it will always try to verify that signature. This is in contrast to the Windows loader, which will verify the signature of a file only in specific instances, such as when the file is an ActiveX control. This verification can be quite time intensive, since it can require hitting the network several times to download up to date certificate revocation lists, and also to ensure that there is a full chain of valid certificates on the way to a trusted root. So, when an Authenticode signature is applied to an assembly it's not unheard of to see a several second delay while that assembly is being loaded.

In simple words , for any assembly to run we have something called evidences (authenticode can be one) which will be used to evaluate the policy or permissions that the code has to execute . In case if your exe already has full trust then we should be good to add this entry into the config file .

To know if your exe has fulltrust please run the following command.
Go to C:\Windows\Microsoft.NET\Framework\v2.0.50727

From there run , caspol –all –lf >>hi.txt

Now in hi.txt try to search for your assembly . If your assembly is there then that means it has full trust.

For more
info on security and caspol , please refer the following blogs/articles.

msdn.microsoft.com/en-us/library/ff648663.aspx

msdn.microsoft.com/en-us/library/cb6t8dtz(v=vs.110).aspx

Keep Coding!
Nandeesh Swami
Visual Studio Languages and Runtime