64-bit Managed Custom Actions with Visual Studio

A reader who happened across my post on
Installer on 64-bit Platforms

a problem with running 64-bit managed custom actions using the
Visual Studio 2005 Windows Installer project. This also recently cropped up in
an internal discussion alias.

The issue is that if you build a managed class library project targeting a 64-bit platform
using /platform:x64 or /platform:Itanium and install a
Windows Installer package built in Visual Studio 2005 on a
64-bit machine a
System.BadImageFormatException is thrown. The reason is because
the native shim packaged with the .msi file is a 32-bit executable.

Let’s step back a minute, though, to how to build a Windows Installer setup
project with managed custom actions. I won’t go into
details, but basically you
create a new Class Library project that contains one or more derivatives from

System.Configuration.Install.Installer class
. In the Custom
Actions editor for your Windows Installer project you can right-click on a
specific phase (Install, Commit, Rollback, or Uninstall) or, preferably, the root node
(which adds the custom action to all phases with the appropriate custom action
types) and add whatever you want from
project output to a specific file in your file system. If your class library is
in the same solution I recommend clicking “Add Output” in the custom action
editor dialog.

You should also click on the Windows Installer project and change the
TargetPlatform property to either x64 or Itanium, depending on what you’re
targeting. This makes sure that 64-bit components are installed to the 64-bit
folders like [ProgramFiles64Folder]. If you don’t set this according to what
binaries you’re installing (which can be a mix of both 32- and 64-bit) 64-bit
files will be installed into [ProgramFilesFolder] which, on 64-bit platforms,
is, for example, C:Program Files (x86).

Back to the problem. When you build the Windows Installer project in Visual
Studio 2005 it embeds the 32-bit version of InstallUtilLib.dll into the

Binary table
as InstallUtil. When Windows Installer executes your managed
custom action it actually is calling the ManagedInstall entry point function
from InstallUtilLib.dll as a
type 1 deferred custom action (1025) which creates
an instance of the CCW
System.Configuration.Install.IManagedInstaller interface
and runs your Installer classes. Since the native
is 32-bit it loads the 32-bit Framework which will throw
the BadImageFormatException since your managed class library is 64-bit.

To workaround this issue you either need to import the appropriate bitness of
InstallUtilLib.dll into the Binary table for the InstallUtil record or –
if you do have or will have 32-bit managed custom actions add it as a new record
in the Binary table and adjust the

CustomAction table
to use the 64-bit Binary table record for 64-bit managed
custom actions.

To replace the 32-bit InstallUtilLib.dll with the 64-bit bitness,

  1. Open the resulting .msi in Orca from the Windows Installer SDK
  2. Select the Binary table
  3. Double click the cell [Binary Data] for the record InstallUtil
  4. Make sure “Read binary from filename” is selected and click the Browse
  5. Browse to %WINDIR%Microsoft.NETFramework64v2.0.50727
  6. Select InstallUtilLib.dll
  7. Click the Open button
  8. Click the OK button

Note that the Framework64 directory is only installed on 64-bit platforms and
that it corresponds to the 64-bit processor type. That is, you won’t find the
x64 flavor of InstallUtilLib.dll on an IA64 machine.

If you already have or anticipate having 32-bit custom actions in future
patches – and I recommend this approach because the future is difficult to
predict – you should add a new record.

  1. Open the resulting .msi in Orca from the Windows Installer SDK
  2. Select the Binary table
  3. Click the Tables menu and then Add Row
  4. Enter, for example, InstallUtil64 for the Name
  5. Select the Data row and click the Browse button
  6. Browse to %WINDIR%Microsoft.NETFramework64v2.0.50727
  7. Select InstallUtilLib.dll
  8. Click the Open button
  9. Click the OK button
  10. Select the CustomAction table
  11. For each custom action where the Source column is InstallUtil and only those
    custom actions that are 64-bit managed custom actions (or that were built with /platform:anycpu,
    the default, where you want to run as 64-bit custom actions), change the value
    to, for example, InstallUtil64

This only affects DLLs build with /target:library. Managed EXEs will run
correctly according to what platform they target.