After upgrading from IIS 5 to IIS 6, client observed the web application performance was not as good as before. This site uses Windows Integrated Authentication, by compare the IIS log, customer found huge number of 401s were recorded in IIS 6 log. And each 401 takes about 300+ms since customer's network has high latency.
Further investigation found this related to keep-alive. IE always uses 2 connections when accessing the old web site. However, for new site, IE opens lots of connections instead. For every new connection, IE sends out anonymous request first, and then sends another request with authentication information after received the 401 response. For a better details regarding authentication process, see KB264921.
264921 How IIS authenticates browser clients
According to HTTP protocol, persistent connections are the default behavior for HTTP 1.1. That is, unless otherwise indicated, the client SHOULD assume that the server will maintain a persistent connection, even after error responses from the server. So, IE's behavior doesn't conform to the HTTP protocol.
Working with IE engineer, we finally found the cause. This was caused by RemoveServerHeader feature of UrlScan installed on this server. UrlScan is a security tool that restricts the types of HTTP requests that Internet Information Services (IIS) will process. By blocking specific HTTP requests, the UrlScan security tool helps prevent potentially harmful requests from reaching the server. RemoveServerHeader feature allows UrlScan to remove or alter the identity of the server from the "Server" response header in the response to the client.
IE 7 handles the response incorrectly if the "server: Microsoft IIS/6.0" was not found in response headers. This behavior was changed in IE 8 and IE 9. The workaround is disabling the RemoveServerHeader feature of UrlScan.
RemoveServerHeader feature offers no real security benefit. Most HTTP attacks are not operating system specific. Also, it is possible to detect the identity of a server and information about the operating system by mechanisms that do not depend on the server header. So, don't worry that this may reduce your web server's security level.
HTTP/1.1, part 1: URIs, Connections, and Message Parsing
See you next time,