Of Web services and client-side certificates

I recently had a customer embark upon the process of securing Web services by using SSL and client-side certificates for authentication.  On the surface, this seems a very straightforward process, but after spending four days of questioning why four simple lines of code refused to work, I came to understand one of the rougher edges in the .NET Framework.  Furthermore, I could find nothing that documented this behavior nor could I find any references to this particular problem on the Internet.  What follows is my contribution to all of you who program by example in hopes that you’ll stumble upon this in your quest to make client-side certificates work for authentication.

According to all of the samples you can find within the documentation for the .NET Framework and on the Internet, the following two lines of code will create an X509 certificate and associate it with Web services proxy:

X509Certificate cert = X509Certificate.CreateFromCertFile(@"c:\kevinha.cer");
proxy.ClientCertificates.Add(cert);

From here, it would seem that all subsequent calls made over SSL with this given proxy will use the certificate specified in kevinha.cer for authentication.  Alas, things are not quite that simple. At first glance, you’ll notice that cert seems to contain the proper representation of the certificate as specified in the c:\kevinha.cer file. In fact, examining the proxy.ClientCertificates collection does indeed show that a certificate is present. However, there are two more out-of-band items that are required for your certificate to be sent. Failure to meet these two requirements will result in silent failure.

First, the certificate with the private key must be imported into your personal store. Unfortuantely, things are a little more complicated if you’re running as a service, an ASP.NET application or a COM+/Enterprise Services component. If you’re in the latter boat, there are pleanty of examples and whitepapers on msdn.microsoft.com that outline how to load a profile such that the certificate store is accessible. Let me say it again, the certificate with the private key must be imported into your personal store. On first glance, you may think that the certificate file on disk is all you need, but think about it for just a moment … for identification with X509 certificates to work, you the client must have access to your private key. In all liklihood, you created the .cer file by exporting your certificate from the certificate store, but when you did this you did not include the private key. When you do include the private key, you’re prompted to protect the private key with a password and you end up with a .pfx file. Of course, you might think that you could use the .pfx file and have everything you need, but as you’ll soon find out, that doesn’t work either.

So, under the covers the .NET Framework uses the identifying information the certificate loaded from the .cer file to find full certificate with the private keyin your personal certificate store.  It is here that the second problem rears its ugly head …

When you import your certificate into the certificate store, you are presented with the option to “Enable strong private key protection.  You will be prompted every time the private key is used by an application if you enable this option.”  Unfortuantely, enabling this option causes the .NET Framework to silently fail when accessing the private key of your certificate.  From within Internet Explorer, a dialog box is presented informing you that IE is accessing a certificate, permits you to chose the certificate to use for identification and proceeds as one would expect.  The .NET Framework, however, never presents such a dialog.  In fact, if you are building server-side applications you do not want such a dialog to be displayed as there is no interactive session to respond to the dialog.  And it is here that I spent four days of hell wondering why my certificates were not being sent.  Should you enable this option, you will never see a dialog nor will you ever see an exception indicating that the certificate could not be accessed.  Instead, you will most likely see an HTTP 403 error manifest itself as an exception during the actual invocation of the Web method.

Now, why can’t we use the .pfx file which contains the certificate with the private key and used X509Certificate.CreateFromSignedFile?  Turns out that this isn’t really supported today.  If you look at the method in detail, no where does it require the password necessary to access the certificate stored within the .pfx.  Remember, when you export your certificate, the private key is protected with a password.  Since CreateFromSignedFile has no arguements for the password, you are effectively at a roadblock.  Techhnically speaking, CreateFromSignedFile isn’t a supported method for using client-side certificates for authentication.

So, the lesson learned here is do not eanble strong private key protection for certificates you plan to use programatically from the .NET Framework.