More details about how .NET Framework 2.0 setup reduces system reboots

A while back, Quan To posted an item on his blog that briefly described a project he worked on to help reduce the number of reboots in the .NET Framework 2.0 setup.  Recently, I learned more details about how this feature works, and I wanted to post them here in case anyone is curious.

.NET Framework 2.0 setup carries an updated copy of the file mscoree.dll.  This file is shared between all versions of the .NET Framework and installs to %windir%\system32, which means that if the file is in use because there is a .NET Framework 1.0 or 1.1 application running on the system, it could potentially cause a reboot at the end of .NET Framework 2.0 setup.

The reboot reduction algorithm for the .NET Framework 2.0 looks like the following:

Windows Installer uses the MoveFileEx API behind the scenes to copy the new version of mscoree.dll.  If mscoree.dll is in use, Windows Installer will report back return code 3010 (which means success with a reboot required).

The external UI handler for the .NET Framework 2.0 intercepts the 3010 return code and performs the following additional checks:

  • Is the reboot request caused by mscoree.dll (and only by mscoree.dll)?
  • Is mscoree.dll soft-locked?  If it is soft-locked (as opposed to hard-locked), then this means that MoveFileEx was able to rename mscoree.dll and copy the new version to %windir%\system32.  If it is hard-locked, MoveFileEx would not be able to rename mscoree.dll and a reboot would be necessary to enable copying the new version to %windir%\system32.

Note that in my experiences with hard-locking tests, Windows Installer will fail to copy the file and prompt you to abort, retry or cancel and there isn't a way to get a successful installation when a file that needs to be updated is hard-locked.  As an example of this type of scenario, you can install the .NET Framework 1.1, open mscoree.dll in Microsoft Word then try to install the .NET Framework 2.0.

If both of the above conditions are true, the external UI handler changes the 3010 return code to 0 and does not prompt the user to reboot.  In this scenario, any applications that were previously running will have the old version of mscoree.dll loaded into their process and continue to work correctly.  Any new applications that run after installing the .NET Framework 2.0 will be able to use the updated version of mscoree.dll now located in %windir%\system32.

An interesting point to keep in mind here is that this logic to intercept and override the 3010 reboot return code is located within the external UI handler for the .NET Framework 2.0 setup.  This means that you will not get this feature when deploying the .NET Framework 2.0 using the MSI directly.  In that scenario, Windows Installer will return 3010 whenever setup completes in cases where mscoree.dll is in use during installation, and the code that modifies this return code does not ever get run.  However, applications that include the .NET Framework 2.0 as a prerequisite can safely defer the reboot until the end of the overall application installation process (as opposed to requesting a reboot in the middle of installation).