Invalid algorithm specified when signing with RSACryptoServiceProvider and SHA 256

Hi all,

The other day a customer of mine was getting the following error when signing with RSACryptoServiceProvider after specifying SHA 256 as hash algorithm: invalid algorithm specified. He was using a non-Microsoft CSP (Cryptographic Service Provider).

We checked the CryptoAPI calls that .NET was calling behind the scenes (you may do that with my CryptoAPI Tracer script, for instance). We saw that CryptCreateHash API was returning the following error: 0x80090008 (Invalid algorithm specified), and we were passing the proper Algorithm Id to it: 0x800c (We can see in ALG_ID that this id corresponds to CALG_SHA_256).

Now, why was CryptCreateHash failing then?

Note that CryptoAPI doesn’t perform crypto operations by itself. It redirects the parameters that it gets from our application to the desired CSP, and the CSP performs the operations on its behalf. So the error we are getting is actually coming from the CSP.

So summing up, .NET calls CryptoAPI with proper parameters, CryptoAPI redirects the call to the third-party CSP, but that CSP doesn’t recognize CALG_SHA_256 as one of the algorithms it supports. There is no issue with .NET code or CryptoAPI code. The CSP just doesn’t support SHA 256.

Remember that SHA 256 support depends on the CSP you are using, like I explained on this post (which also applies to versions of Windows newer than XP): SHA-2 support on Windows XP.

In this case, the only solution is to talk to the providers of the CSP to ask for SHA 256 support, or change to another CSP.

I hope this helps.

Regards,

Alex (Alejandro Campos Magencio)