IIS 6 applications may stop responding after you install Microsoft update KB 982666

In the last few days we have started seeing a new symptom appearing more and more often on IIS 6 servers. Here are the symptoms:

You are running Windows 2003 Server and you have IIS 6 installed and hosting your web-sites. The operating system prompts you to install Microsoft update MS10-040 (KB 982666). After completing the install you note the following errors:

  • IIS application pools or websites will no longer start
  • IIS web-sites may not be able to start.
  • Rapid Fail Protection will shut down your application pools 
  • An inspection of the event logs show that the IIS worker processes are terminating unexpectedly, showing event messages similar to the following:

Event Type: Warning
Event Source: W3SVC
Event Category: None
Event ID: 1009
Date: 12/9/2009
Time: 10:55:01 AM
User: N/A
Computer: WEBSERVER01
Description:
A process serving application pool 'DefaultAppPool' terminated unexpectedly. The process id was '1234'.
The process exit code was '0xffffffff'.

Ensure that you are seeing the above process exit code '0xffffffff' for which this post applies.

So what really happens?

NOT ALL IIS installs running on Windows 2003 will be impacted by this patch. YOU CAN INSTALL the patch and continue running IIS normally. The only case when the error occurs is the following:

You installed Windows 2003 and you did not install IIS 6 when you did this. You then installed Service Pack 1 or Service Pack 2 directly and afterwards you decided to install IIS. When installing IIS you were prompted to insert the original installation disk (not the Service Pack 1 disk or the Service Pack 2 disk).

If you followed the scenario above, you will most likely end up with what is called a binary mismatch between the operating system dlls and the dlls that belong to IIS. In plain English: your Windows 2003 files are those of Service Pack 2 and your IIS files are those of Service Pack 1 or RTM.

How can you verify that you are in this scenario?

In a Windows 2003 command prompt, type the following command:

Winver

This will show you the Windows 2003 version that you are running and the patch level. You should see that the operating system is patched to Service Pack 2. If you are not, you should seriously consider upgrading since Service Pack 1 of Windows 2003 is no longer supported.

If you are running service pack 2, then go ahead and check the version of an IIS 6 files in the following folder: %windir%\system32\inetsrv. The minimum version of the file(s) is the following if you are running SP2.

6.0.3790.3959

What goes on behind the scenes?

The latest Microsoft update for Windows 2003 server updates the w3core.dll. This new version of the file will try and call an API located in iisutil.dll. The invoked API will only be found in the iisutil.dll that is installed by the Service Pack 2 of Windows 2003. If you installed IIS 6 after having installed Windows 2003 Service Pack 2, it is very possible, that the installer used the RTM or Service Pack 1 version of the dll, in which the API is not present. Thus the call fails.

Solution to this:

Should you find yourself in this situation, and only after you confirm the symptoms, you should go ahead and re-install the Service Pack 2 from Windows 2003. To download the Service Pack you can go to:

https://www.microsoft.com/downloads/details.aspx?FamilyId=95AC1610-C232-4644-B828-C55EEC605D55&displaylang=en

You do not need to un-install and re-install Service Pack 2, just re-install it. Upon install, the Service Pack installer will make a list of all Windows components that are installed on your system (including IIS) and will update their binaries.

Note:

This issue is specific to Windows Server 2003. We won’t run into this issue on a Windows 2008 server.

More update to follow on this issue soon…

*Reference/Credit for this post goes to this article for a similar issue earlier in December 2009 where KB973917 was causing similar issues.