Error installing .NET Framework on Windows XP SP2 caused by Data Execution Prevention (DEP)
Note - the issue described in this blog post was originally presented as an issue on Windows XP SP2. However, it can also affect .NET Framework 1.0 and 1.1 installation on any OS released after the .NET Framework 1.0 and 1.1 shipped - specifically, I have seen reports of this issue on Windows Vista. The steps listed here are applicable to this type of install failure on other newer OS's like Windows Vista and not just Windows XP SP2.
As I was researching the bug in .NET Framework 1.0 and 1.1 that is related to regional language settings on Windows XP SP2 (see this blog post for full details), I discovered an additional issue introduced in Windows XP SP2 that can cause .NET Framework 1.0 or 1.1 setup to fail. Just like the other bugs, this one causes .NET Framework setup to report an error while registering System.EnterpriseServices.dll. In the case of .NET Framework service pack setup, it will cause an error to occur while extracting the service pack to a temporary location before even starting setup.
Windows XP SP2 introduced a new security feature known as Data Execution Prevention (or DEP for short). There is a good article introducing DEP here. If a computer running XP SP2 has hardware that supports DEP and DEP is enabled in boot.ini, installation of the .NET Framework will fail while trying to register System.EnterpriseServices.dll (which happens to be the first time that managed code gets run during setup and therefore is the first time the bug is hit). Also, installation of .NET Framework service packs will fail while trying to extract the service pack setup to a temporary location because the .NET Framework service pack wrapper is written in managed code.
The DEP feature was introduced after the .NET Framework 1.0 and 1.1 shipped and unfortunately the .NET Framework is not compatible with DEP without applying a service pack. Like the language settings bug, this DEP compatibility bug has been fixed in the .NET Framework 1.0 SP3 and 1.1 SP1. However, this bug causes the initial installation of the .NET Framework to fail and rollback, and you cannot install the service pack without first getting the product installed (unless you use a method like I describe here, which will work but is not "officially" supported).
If you are running into this bug on your Windows Vista, Windows Server 2008 or Windows 7 computer, you can use steps like the ones in this blog post to work around this bug in the .NET Framework 1.0 and 1.1 and get it installed.
If you are running into this bug on your Windows XP SP2 computer, you can use the following steps to work around this bug in the .NET Framework 1.0 and 1.1 and get it installed:
- From the Start menu, type sysdm.cpl in the Run box or go to Control Panel and choose the System item
- Click the Advanced tab
- Click the Settings button in the Startup and Recovery section of this tab
- In the Default operating system dropdown, select each option that starts with "Microsoft Windows XP Professional" and change /NoExecute=Optin to /NoExecute=AlwaysOff. This will update the settings in boot.ini on the computer
- Click OK
- Restart
- Install the .NET Framework 1.0 or 1.1
- Install .NET Framework 1.0 SP3 or 1.1 SP1
- Return to the System control panel, select each option that starts with "Microsoft Windows XP Professional" and change /NoExecute=AlwaysOff back to /NoExecute=Optin
For reference, here is what the Startup and Recovery item in the Advanced tab of the System control panel looks like (this is the screen you will see in step 4 of the instructions above):
<update date="4/17/2008"> Added a note indicating that the issue in this post can affect the .NET Framework 1.0 and 1.1 setup on Windows Vista and not just Windows XP SP2 </update>
<update date="6/23/2009"> Fixed broken image link, and added a link to a separate blog post that provides steps for turning DEP on and off on Windows Vista and higher. </update>