Test Signing and Instrumentation
Boy, I seem to spend all my time in the deep dark recesses of the CLR. OK, this post is a little bit out there; I hope someone finds it useful.
There's a problem with doing instrumented profiling of applications in Visual Studio 2005 Team System that comes up when working with strongly named assemblies. The problem is quite simply that instrumentation breaks the strong name signature of your assemblies. And in working around this problem, I'll show you how you can actually be hiding start-up performance problems in your .NET applications.
Let us recap how assembly signing works in .NET. First, you create a public/private encryption key pair using sn.exe -k or similar. Then you take your assemblies files and you sign them. This signing can be done with the sn.exe tool or more transparently by using the /keyfile or /keycontainer command line options of the C# or VB compilers.
Basically what the signing process actually does is to compute a cryptographic hash from the metadata, intermedate language (IL) and other parts of your assembly file. It then takes this hash and encrypts it using the private key from your key pair. Finally this signature blob and your public key are embedded in the assembly. Now you can send your assembly out into the world safe in the knowledge that no one can mess with it.
Any CLR that your assembly gets presented to can now validate that your assembly has not been tampered with. This is done by first calculating the same hash that the signing process calculated. Then, the CLR takes the signature blob from the assembly and uses the public key to reverse the encryption performed by your private key. This will only work if the public key is truly the partner of the private key used to encrypt the signature. Hence the importance of keeping the private key, uh, private. If all is well, the decrypted signature and the calculated hash will match, and Bob's your Uncle!
Note: The 16 character public key token that you see as part of the full name of a strongly named assembly is not the signature hash referred to above. It is just a hash of your assemblies public key which makes the key a little easier to work with. For example, the public key token
7e16da7b2e1bcfa6
is merely a hash of the public key
0024000004800000940000000602000000240000525341310004000001000100e91eb19e092741
e85e7eda996ba566d491be9a300b2c26a8ce015398c87529f76075385c960bbc4dcf24ece0d1ad
a9f312dd0a42538d490be9228148645ecf5f45aadc12404af4c07d4b8060056b4724174ecc7d65
c65ac455a7321f71adfa8be981b02e9c5bd721d588395fa212c4a79cc007cf3a60738a0777614e
15b990a5
Which would you rather type?
So signing allows tampering to be caught, but it doesn't prevent it. And there's a back door into the signing process using the sn tool that allows signature verification to be turned off for assemblies containing a specific public key. In VSTS performance (and coverage) instrumentation works by inserting additional calls at the basic block boundaries in your applications IL and then rewriting the application binaries to disk. We call this static instrumentation because it's done before the program runs. Thus it breaks strong names signatures by changing the hash that will be computed when the assembly is verified by the CLR.
OK, big deal right? What's all this got to do with performance profiling? Well, the issue is that validating assembly signatures takes time. All that cryptographic stuff is computationally expensive. So, if you turn off verification for an assembly using something like:
sn -Vr *,b5f241dcf0cb58c4
Then you are potentially hiding a performance problem that might occur when assemblies are fully signed and signing is not disabled. Now, before we go running screaming into the hills, there's one caveat that has to do with the the global assembly cache (GAC). Putting an assembly in the GAC requires that the assembly be strongly named. The full signature verification for assemblies in the GAC is actually only performed when the assembly is inserted into the GAC, and not on subsequent loads of the assembly. So in that sense, there's actually a small performance boost to be had by putting assemblies in the GAC. But all other strongly named assemblies will incur a verification check the first time they are loaded into an AppDomain.
For what types of applications might turning off strong naming incur an artificial performance boost? Well, in practice this is only likely to be a problem for applications that depend on very large numbers of strongly named private assemblies, and even then it's only going to skew the applications start-up time. Applications that load and unload assemblies via additional AppDomains might also incur some penalty, but then I suspect that other factors such as memory fragmentation and disk I/O are much more likely to cause performance noise that will hide this issue.
So, what is test signing? Test signing is a halfway point between fully signed/fully verified and unsigned/unverified. It's a new feature in .NET 2.0. With test signing you can create a throw away test key that you can use to sign your assemblies. The CLR verification process then goes through the full set of computations to verify the assembly hash against the test public key, resulting in the same performance hit as if the assembly were fully signed. However, the assembly doesn't actually contain the public key for the test key pair, but the real public key for the assembly. In this way the strong name of the assembly is not broken. The public key for the test key pair actually lives in the registry.
OK, time for an example. Let's say you just generated a new project called ConsoleApplication1. Let's say that it starts of fully signed using your companies real key pair. Now we instrument:
>vsinstr ConsoleApplication2.exe
Microsoft (R) VSInstr Post-Link Instrumentation 8.0.50727.0
Copyright (C) Microsoft Corp. All rights reserved.
File to Process:
C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe
Original file backed up to C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe.orig
Warning VSP2001 : C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe is a strongly
named assembly. It will need to be re-signed before it can be executed.
Successfully instrumented file C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe.
Warning VSP2013 : Instrumenting this image requires it to run as a 32-bit process. The CLR header
flags have been updated to reflect this.
Note that the message about having to sign the assembly before it can be executed, should really say something like "before it can be strong name verified". Now you create your test key and test sign the assembly.
>sn -k TestKey.snk
Microsoft (R) .NET Framework Strong Name Utility Version 2.0.50727.0
Copyright (c) Microsoft Corporation. All rights reserved.
Key pair written to TestKey.snk
And you'll need just the public key too, so lets get that now:
>sn -p TestKey.snk TestPublicKey.snk.
Microsoft (R) .NET Framework Strong Name Utility. Version 2.0.50727.0
Copyright (c) Microsoft Corporation. All rights reserved.
Public key written to TestPublicKey.snk
You need the public key token that's currently in the assembly, which remember is from the real public key pair and is part of the real strong name of the assembly.
>sn -T ConsoleApplication2.exe
Microsoft (R) .NET Framework Strong Name Utility Version 2.0.50727.0
Copyright (c) Microsoft Corporation. All rights reserved.
Public key token is b5f241dcf0cb58c4
Now you do the test signing:
>sn -TS ConsoleApplication2.exe TestKey.snk
Microsoft (R) .NET Framework Strong Name Utility Version 2.0.50727.0
Copyright (c) Microsoft Corporation. All rights reserved.
Assembly 'ConsoleApplication2.exe' successfully re-signed
And finally you register the test signing public key:
>sn -Vr *,b5f241dcf0cb58c4 TestPublicKey.snk
Microsoft (R) .NET Framework Strong Name Utility Version 2.0.50727.0
Copyright (c) Microsoft Corporation. All rights reserved.
Verification entry added for assembly '*,b5f241dcf0cb58c4'
Sadly the message that the last step gives you is identical to the one you get for regular verification skipping registration. Go to the registry and look under:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\StrongName\Verification\*,b5f241dcf0cb58c4
(or whatever your public key token is). You'll see a TestPublicKey value containing the public key token.
Now you can run your performance tests and measure the real cost of checking all those assembly strong name signatures.