Error installing .NET Framework 1.0 and 1.1 on Windows Vista caused by Data Execution Prevention (DEP)

  • Article

A while ago, I described a .NET Framework 1.0 and 1.1 installation failure that can be caused by the Data Execution Prevention (DEP) feature that was introduced in Windows XP SP2.  My previous blog post about this topic stated that this DEP issue can affect .NET Framework 1.0/1.1 setup on Windows XP SP2.  However, this issue can cause installation failures on all operating systems that include the DEP feature, including Windows Server 2003 SP1 and Windows Vista.

As I described in my previous blog post, the DEP feature was introduced after the .NET Framework 1.0 and 1.1 shipped, and the original releases of the .NET Framework 1.0 and 1.1 are not compatible with DEP.  This DEP compatibility bug has been fixed in the .NET Framework 1.0 SP3 and 1.1 SP1.  However, this issue can cause the initial installation of the .NET Framework 1.0 and 1.1 to fail and rollback, and you cannot install the service pack without first getting the product installed (unless you use a method like creating a combined administrative install point, which will work but is not "officially" supported).

To workaround this issue, you can temporarily disable DEP, install the .NET Framework 1.0 or 1.1 and then re-enable DEP.  The method of disabling DEP on Windows XP SP2 is described in the previous blog post, but it has been changed in Windows Vista.

The following steps can be used to disable Data Execution Prevention (DEP) on Windows Vista:

  1. Click on the Start menu, choose All Programs, then Accessories, then right-click on the Command Prompt item and choose Run as administrator
  2. Click Continue to grant permission to launch a cmd prompt with administrator privileges
  3. Run the command bcdedit.exe /set nx AlwaysOff
  4. After running the command, you should see a message in the cmd prompt stating The operation completed successfully
  5. Restart the computer

The following steps can be used to re-enable Data Execution Prevention (DEP) on Windows Vista:

  1. Click on the Start menu, choose All Programs, then Accessories, then right-click on the Command Prompt item and choose Run as administrator
  2. Click Continue to grant permission to launch a cmd prompt with administrator privileges
  3. Run the command bcdedit.exe /set nx OptIn
  4. After running the command, you should see a message in the cmd prompt stating The operation completed successfully
  5. Restart the computer

Note - this issue will not affect all Windows Vista systems.  DEP is hardware-enforced and not all CPUs are compatible with it.  However, if you are having trouble installing the .NET Framework 1.0 or 1.1 on Windows Vista and installation fails while trying to register System.EnterpriseServices.dll, it is worth trying the above workaround in case you do have a DEP-compatible system.

<update date="11/13/2008"> Updated bcdedit.exe command lines slightly. They used to say bcdedit.exe /set {current} nx AlwaysOff and bcdedit.exe /set {current} nx OptIn. However, the {current} switch can be omitted and bcdedit.exe will automatically apply the setting change to the boot entry for the currently running OS. </update>