Trying to force SharePoint servers to use only FIPS 140-2 compliant algorithms is not an uncommon event. It's admittedly tempting. But invariably it breaks something in SharePoint 2010. It can and will cause one or more big problems for the proper functioning of SharePoint 2010. The most notable one is clients seeing "503 Service Unavailable" when attempting to browse to a SharePoint site. If that happens, the entire SharePoint farm becomes mostly worthless. Also workflows may fail to run on a SharePoint server that is locked down in this way. Furthermore, even workflows that are created on a machine that is locked down in this way may not run on a SharePoint server or in Windows Workflow Foundation. WCF services may break too. Infopath forms services may possibly break as well.
Hundreds of people have already learned the hard way that FIPSAlgorithmPolicy and SharePoint 2010 (and MOSS 2007) don't mix. Hopefully this blog post will help prevent others from having to go through the same pains.
Also worthy of note is that the encryption algorithms that SharePoint uses to communicate with between its servers and clients are FIPS compliant. The few examples documented where SharePoint 2010 does not use FIPS compliant algorithms have nothing to do with external communication—just "internal processing."
Primary Sources of Documentation
Workflows fail to start when the Federal Information Processing Standard (FIPS) algorithm is enabled on a SharePoint server. . . SharePoint Server uses several Windows encryption algorithms that do not comply with Federal Information Processing Standard (FIPS) 140-2 - Security Requirements for Cryptographic Modules. For example, SharePoint Server 2010 uses MD5, which does not comply with FIPS 140-2, for computing hash values that are not used for security purposes.
SharePoint Server 2010 uses several Windows encryption algorithms for computing hash values that do not comply with Federal Information Processing Standard (FIPS) 140-2, Security Requirements for Cryptographic Modules. These algorithms are not used for security purposes; they are used for internal processing. For example, SharePoint Server 2010 uses MD5 to create hash values that are used as unique identifiers. Because SharePoint Server 2010 uses these algorithms, it does not support the Windows security policy setting that requires FIPS compliant algorithms for encryption and hashing. . . . The Windows FIPSAlgorithmPolicy registry key is. . . a useful enforcement tool for many solutions, but not [for] SharePoint Server 2010.
The application may crash, Visual Studio may crash, or you may receive an error message when you develop or run a Windows Workflow Foundation-based application on a computer that has FIPS-compliant algorithms enabled. When you develop or run a Microsoft Windows Workflow Foundation-based application on a computer that has Federal Information Processing Standard (FIPS) compliant algorithms enabled, you may experience. . . symptoms. This issue occurs because Windows Workflow Foundation uses the MD5CryptoServiceProvider class to provide non-secure hashing of a string to a unique key. The MD5CryptoServiceProvider class does not support FIPS compliance.
Secondary Sources of Documentation
.NET 2.0 introduced a check for FIPS certified algorithms if your local security policy was configured to require them. This resulted in algorithms which are not FIPS compliant (or implementations which were not FIPS certified) throwing an InvalidOperationException from their constructors. In some cases this isn't a desirable behavior. For instance, some applications need to use the MD5 hashing algorithm for compatibility with an older communication protocol or file format. Prior to .NET 3.5, the AES algorithm was only available in an implementation which was not FIPS certified, and if you needed to use that algorithm the FIPS check could also block you. To help these cases, we added a configuration file switch to .NET 2.0 SP 1 (and therefore .NET 3.5) which allows an application to say "I know what I'm doing, please don't enforce FIPS for me".
[Note that SharePoint 2010 uses .net 2.0/3.5.]
You may receive an error message when you access ASP.NET Web pages that have ViewState enabled after you upgrade from ASP.NET 1.1 to ASP.NET 2.0. . . This problem occurs when the following conditions are true: •The HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\fipsalgorithmpolicy registry subkey is set to 1. •ASP.NET 2.0 uses the RijndaelManaged implementation of the AES algorithm when it processes view state data. The ReindaelManaged implementation has not been certified by the National Institute of Standards and Technology (NIST) as compliant with the Federal Information Processing Standard (FIPS). Therefore, the AES algorithm is not part of the Windows Platform FIPS validated cryptographic algorithms.
[Note that SharePoint 2010 uses asp.net 2.0/3.5. It may not use the RijndaelManaged implementation of the AES algorithm, however.]
This problem occurs because the MD5 algorithm is not FIPS compliant. The MD5 algorithm uses Windows Communication Foundation to obtain a hash value. The hash value generates a unique name for a data contract.
[Note that SharePoint 2010 uses WCF. Although it may not use this specific WCF call or need this fix.]
Tertiary Sources of Documentation
Standard blog disclaimers apply.
Additional Search Key Words
HKLM\System\CurrentControlSet\Control\Lsa\FIPSAlgorithmPolicy\Enabled HKLM\System\CurrentControlSet\Control\Lsa\FIPSAlgorithmPolicy FIPS compliant algorithms for encryption, hashing, and signing Group ID (Vulid): V-3383 The system is not configured to use FIPS compliant Algorithms for Encryption, Hashing, and Signing. Rule Version (STIG-ID): 3.077 Configure the system to require the use of FIPS compliant algorithms. Navigate to Local Policies -> Security Options. If the value for "System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing" is not set to "Enabled", then this is a finding.