Inside Internet Explorer’s Tools > Internet Options > Advanced dialog, there’s an option named Enable Integrated Windows Authentication:
This preference is stored using a REG_DWORD named EnableNegotiate inside HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings. When set to 0x1, Negotiate is enabled.
From time to time, users have asked what this option does and why they’d ever want to adjust it. The answer is that the Integrated Windows Authentication (IWA) option controls whether Internet Explorer (and applications based on WinINET) will use the Negotiate authentication protocol to respond to HTTP/401 challenges from servers. At a high-level, Negotiate is a “wrapper” around the Kerberos and NTLM authentication protocols. If Kerberos can be used, it will be, otherwise the Negotiate protocol will use NTLM. If you disable the IWA protocol, then a HTTP/401 challenge which only offers Negotiate will be treated as “final” and the client will refuse to authenticate.
Most servers are configured to return a challenge which offers both Negotiate and “unwrapped” NTLM, such that if the user has disabled Negotiate by unticking the IWA checkbox, the client will still be able to authenticate using the NTLM protocol:
Fiddler’s Auth inspector tabs will show you what authentication protocol is in use:
…and when NTLM is in use, it will even break down the authentication blobs into a textual explanation:
IWA is enabled by default and generally should be left that way. However, for troubleshooting purposes, it is sometimes useful to disable the option (don’t forget to restart all IE instances) to see whether a given authentication problem may be related to the use of Kerberos.
For instance, some users have encountered problems where a given server returns a HTTP/400 error when IWA is enabled:
HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Date: Wed, 06 Jul 2011 20:48:07 GMT
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Request Too Long</h2>
<hr><p>HTTP Error 400. The size of the request headers is too long.</p>
If IWA is disabled, then NTLM is used and the request succeeds. The problem in this case is that many IIS servers are configured to refuse HTTP requests that contain more than 16kb of header information. A Negotiate authentication string contains a base-64-encoded Kerberos ticket which includes a list of all of the security groups to which the current user belongs. In some environments this list can grow quite large and hence the client’s Authorization header alone exceeds the 16kb limit. When IWA is disabled, NTLM is used instead and the smaller Authorization token does not exceed the configured limit.
A member of the IIS Team wrote a blog further explaining this issue, including steps to see how many groups you belong to. IIS servers can be configured to allow larger HTTP requests, which works around this problem without reconfiguring the client or trimming the user’s group memberships. Servers can also be reconfigured to permit connection-based Negotiate authentication which lowers the performance impact of sending large Kerberos tickets on every request. You can also read this KB.
PS: See this post for a great list of troubleshooting steps for Kerberos authentication.