Why does .NET Framework 3.5 beta 1 setup tell me to install Windows XP SP2 on Vista?

Recently, we have heard from a few customers who have been trying to install the .NET Framework 3.5 beta 1 or Visual Studio codename Orcas beta 1 on Windows Vista, but received an error message when they launched setup that stated that they need to install Windows XP SP2.  We have found a scenario where this can end up happening, and I wanted to post a description of the problem and a possible workaround in case you end up in this state on your Windows Vista system.

Description of the problem

The .NET Framework or Visual Studio both require Windows XP SP2 before they will allow installation on Windows XP operating systems.  However, if setup is run in Windows XP compatibility mode on Windows Vista, it will think that it needs to run Windows XP OS prerequisite checks and will not be able to find Windows XP SP2 installed.

How the problem can occur - option 1

There are a couple of different ways that .NET Framework or Visual Studio setup can be run in compatibility mode on Windows Vista.  The first way is if setup fails or if you cancel setup, and then you select the option to attempt to reinstall using recommended settings after setup exits.

On Windows Vista, if a setup package exits without creating or removing an Add/Remove Programs entry, Windows Vista assumes that it failed and triggers a Program Compatibility Assistant dialog that states that the program might not have installed correctly.  This dialog has 2 buttons on it - one will cause Windows to try to reinstall using recommended settings, and the other states that the program installed correctly and will not cause Windows to try to take any corrective actions.  This dialog looks like the following:

If you see a dialog like the following after exiting the .NET Framework or Visual Studio setup on Windows Vista, and you select the option to reinstall using recommended settings, that will cause Windows Vista to re-run the setup in Windows XP compatibility mode.  Doing this will then show a block dialog stating that you need to install Windows XP SP2.  To make matters worse, Windows Vista stores this choice in the registry so that if you try to run the same setup package from the same location in the future, it will continue to display the Windows XP SP2 block dialog.

How the problem can occur - option 2

The second way that .NET Framework or Visual Studio setup can be run in compatibility mode on Windows Vista is if you specify this setting directly in the properties for the setup file by doing the following:

  1. Right-click on the setup package and choose Properties
  2. Click on the Compatibility tab in the properties window
  3. In the Compatibility mode section, check the box that is labeled Run the program in compatibility mode for Windows XP (Service Pack 2)
  4. Click OK to save the changes
  5. Re-run the setup package

The .NET Framework and Visual Studio setup packages do not support running in compatibility mode on Windows Vista, so if you have done the above, you should try to uncheck the box labeled Run the program in compatibility mode for Windows XP (Service Pack 2) and re-run setup to see if it resolves this issue.

How to resolve the problem

In order to prevent Windows Vista from running the setup in Windows XP compatibility mode each subsequent time, you can use the following steps to turn off compatibility mode for the setup package:

  1. Right-click on the setup package and choose Properties
  2. Click on the Compatibility tab in the properties window
  3. Click the button at the bottom that is labeled Show settings for all users and click Continue to allow the process to elevate
  4. In the Compatibility mode section, uncheck the box that is labeled Run the program in compatibility mode for Windows XP (Service Pack 2)
  5. Click OK to save the changes
  6. Re-run the setup package

In case you are interested, Windows Vista stores information about the application compatibility mode to use for specific packages in the registry at the following locations:

[HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers]
<full path to setup file> = WINXPSP2

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers]
<full path to setup file> = WINXPSP2

If you have problems clearing the Windows XP compatibility mode setting using the steps listed above, you can also clear this setting by removing the appropriate entries directly from these registry locations.

One last note - in many cases the Program Compatibility Assistant dialog is triggered because of a legitimate failure in either the .NET Framework 3.5 beta 1 or Visual Studio codename Orcas setup.  Once you resolve the compatibility issue where setup tells you to install Windows XP SP2, there will likely be an additional setup error that needs to be resolved.  You can check out my blog categories (for the .NET Framework 3.5 beta 1 and for Visual Studio codename Orcas beta 1) and the Visual Studio Orcas MSDN Forums for additional troubleshooting suggestions if needed.  You can also leave comments on my blog posts or contact me and I can try to help further.

<update date="5/10/2007"> Added information about the HKEY_CURRENT_USER version of the app compat registry value </update>

<update date="5/11/2007"> Clarified the exact logic that Windows Vista uses to trigger a PCA dialog. It monitors additions/removals to the Add/Remove Programs (Programs and Features) control panel, and not the exit code of the process as I originally stated. For reference, see https://www.microsoft.com/indonesia/msdn/appcomp.aspx  </update>