The Case of the App Install Recorder

Adapted from the forthcoming book, Troubleshooting with the Windows® Sysinternals Tools, by Mark E. Russinovich and Aaron Margosis.

A customer had nearly a dozen software packages that wouldn’t install on Windows 7 x64. Every installation program failed immediately with an error message like the one shown in the screenshot below. However, they all installed successfully on 32-bit Windows 7. This error message usually indicates you’re trying to run a 16-bit program, which is not supported on 64-bit Windows versions:

You can use SigCheck to verify the image type. This screenshot shows a set of Seagate Crystal Reports 6.0 installers from 1997 that are 16-bit executables:

The packages all dated from the mid- to late-1990s. Although the packages installed 32-bit Windows components that could presumably run on 64-bit Windows, their installation programs were 16-bit as was often the case back then. 16-bit programs were the only kind that could run on all versions of Windows at the time, particularly on all the CPU architectures that Windows NT supported. Vendors of installer packages therefore used a 16-bit bootstrapper program to detect the operating system version and CPU architecture, and then install the correct binaries for that platform.

However, 64-bit Windows does not include the “NT Virtual DOS Machine” (NTVDM) emulator that enables 16-bit DOS and Windows programs to run on 32-bit versions of Windows NT and its successors, including Windows 7. Wow64 (Win32 emulation on 64-bit Windows) provides a limited ability to emulate some common 16-bit installers, but it didn’t help with these particular installers.  Here’s a 16-bit installer running on 32-bit Windows 7. Note the old “Program Manager” icon in the taskbar, representing the Ntvdm.exe hosting process for 16-bit programs:

After installation, the 32-bit components worked fine on 32-bit Windows 7. It seemed likely that they would probably run fine on 64-bit Windows as well – if they could be installed. All that was needed was a way to replicate the installation steps that were performed on 32-bit Windows on 64-bit.

I came up with a way to record the installation on 32-bit with Procmon, save a filtered trace as XML, then process the XML with PowerShell scripts to capture the resulting file and registry modifications in a way that they could be copied to 64-bit. It identifies only the final names of objects that were moved or renamed, ignores temporary files and objects that were deleted before the installation completed, and excludes system changes made by processes not involved with the installation. The same idea can be used in any other scenario to capture any other types of file and registry key creations or modifications.

To begin, start Procmon, run the installation to completion, then stop the trace (Ctrl+E). Before beginning to apply filters, I recommend saving all events in the trace to a file using Procmon’s native file format as shown in the dialog box below so that you can come back to it later if you need additional data without having to run the installation again.

The next step is to apply filters so that the resulting trace shows only the file and registry operations of interest from the installation-related processes. Open the filter dialog box (Ctrl+L) and add an “Include” rule for each of these operations: CreateFile, WriteFile, SetRenameInformationFile, SetDispositionInformationFile, RegOpenKey, RegCreateKey, RegDeleteKey, RegRenameKey, RegSetValue and RegDeleteValue. Then add a criterion for “Result Is SUCCESS then Include”, as failed operations will not be of interest.

To filter on installation-related processes, open the Process Tree (Ctrl+T). Select the initial installation process as in the screenshot and click “Include Subtree” to set a filter for that Ntvdm.exe process and all its descendant processes.

You should also check to see whether the installation used any out-of-process DCOM components. Such components would run as child processes of the DcomLaunch service, which is hosted in the Svchost.exe instance started with the command line parameters “-k DcomLaunch”. You can inspect the command line of a process in the bottom of the Process Tree window by selecting the process in the tree (see the next screenshot). If any DCOM processes were started while the installation was running, select each and click “Include subtree”. Because it is also possible that an already-running DCOM process responded to a request from the installer, you could also select the DcomLaunch Svchost.exe and click “Include subtree” to include all DCOM processes, although doing so may pick up unrelated system changes.

Finally, if the installer created or modified any services or drivers via the Service Control Manager, the resulting registry changes will have been performed by Services.exe, so select it in the Process Tree and click “Include Process” (not “Include Subtree”). (System changes made by Services.exe may need to be inspected manually later to verify whether they should be captured and “played back”.)

Save the filtered trace to an XML file. Under “Events to save” in the Save To File dialog box, select “Events displayed using current filter” and deselect “Also include profiling events”. Under “Format”, select “Extensible Markup Language (XML)” and deselect “Include stack traces”. The next screenshot shows these options selected. As an option to reduce the size of the XML file, choose Options | Select Columns and show only the Operation, Path and Detail columns before saving the XML file. Save-as-XML saves only the column data selected for display, and those three are all that the script will need.

PowerShell is a particularly adept and flexible tool for manipulating XML, so I wrote a script to read the saved XML and build lists of the new and modified file system and registry objects resulting from the installation and then create a mirrored copy of the file system objects and a RegMods.reg file containing the registry changes that can be imported on another system. Portions of the script are described here; the full script is attached to this blog post and can be downloaded.

The script takes two parameters: the path to the Procmon XML trace, and the path to the target directory in which to build the mirror. For example:

PS C:\Installs> .\Capture-Recording.ps1 .\Crystal-Filtered.XML C:\Installs\Crystal

The script reads the input XML file, and inspects all the events in the trace in the order that they occurred:

# Convert input file into an XML document object
$inputFile = [xml](Get-Content $ProcmonXmlFile)
# Iterate through all the events in the trace
$inputFile.procmon.eventlist.event |
ForEach-Object {

As it processes each event element, the script saves the current element in the variable $ev:

    # Save the current event as $ev
    $ev = $_

It then looks at the event’s Operation and performs the appropriate action based on whether it is a CreateFile, WriteFile, etc.:

    switch($ev.Operation) {

    # File newly created (CreateFile may refer to "read" operations too)
    "CreateFile" {
        # perform actions
    }
    # Existing file modified
    "WriteFile" {
        # perform actions
    }
    # File rename - remove the old name, add the new name
    "SetRenameInformationFile" {
        # perform actions

Processing file system operations is fairly straightforward: for each creation or update event of a file or directory, it adds the event’s path, $ev.Path, to a sorted list of file system objects if the object’s path isn’t already in the list. Similarly, for each deletion event it removes the object path from the list if it’s in the list. Rename events are treated like a delete followed by a create: the old name is removed from the list and the new name added to the list. File system events are ignored if the path is in the user’s temporary directory or appears as a direct write to the user’s registry hive.

The one hitch is that we want to capture only file system modifications, and CreateFile events can be reads or writes. If the saved trace had filtered on “Category Is Write”, then read events would have been filtered out, but that isn’t possible because correct processing of registry operations needs read and write events, as I’ll explain shortly. It would be possible to look at $ev.Category had that column been added to the view prior to saving the XML. But the information we need is also in the Detail column:

        # Verify whether this was a "write" operation
        if ($ev.Detail.Contains("OpenResult: Created") -or
            $ev.Detail.Contains("OpenResult: Overwritten")) {

The Detail column also provides the new object name on a rename operation (the old name is in $ev.Path):

        $ix = $ev.Detail.IndexOf(" FileName: ")
        $newName = $ev.Detail.Remove(0, $ix + 11)

And Detail also confirms whether a SetDispositionInformationFile operation is a file deletion:

        if ($ev.Detail -eq "Delete: True") {

Processing registry events is a little more involved because registry value names can contain backslash characters, unlike the names of registry keys, files or directories which always treat backslashes as delimiters. Procmon captures registry paths as a single text value that can be just a key name or a key name plus a value name. In the latter case it’s hard to determine whether the last backslash is a delimiter between the key and value or part of the value.

To address this issue, the App Install Recorder script tracks all key “open” operations and not just “write” operations. A registry value cannot be accessed until its containing key has been opened, so the script maintains a list of all keys that have been opened, and then looks in that list for the open key whenever a value “write” operation (i.e., RegSetValue or RegDeleteValue) is processed. The script also keeps another sorted list of registry keys in which “write” operations were performed, and each item in that list has its own sorted list of the values that were created or modified within that key.

As with file system operations, “write” operations that create or update keys or values add to the corresponding lists, delete operations remove items from the lists, and renames combine deletes and additions. As with CreateFile, RegCreateKey can also be a read or write operation based on its Detail:

    # A key was (potentially) created; add it to the list of known key names,
    # and add it to the created-keys list if it was created.
    "RegCreateKey" {
        AddOpenKey($ev.Path)
        if ($ev.Detail.Contains("Disposition: REG_CREATED_NEW_KEY")) {
            AddCreatedKey($ev.Path)
        }
    }

After processing each of the events in the Procmon trace, the App Install Recorder script has built sorted lists of all resulting new and modified file system objects and registry data. It builds the mirrored copy of the file system results by iterating through the list of file system objects and copying them to the target location, retaining the directory hierarchy. Capturing the registry changes for playback is more involved. Here the script iterates through the sorted list of written registry keys and runs Reg.exe Export for each key, outputting to a temporary file. It then copies content in the file for the current key only to RegMods.reg, and only for those registry values that had been modified. (This probably isn’t the most efficient way to build a *.reg file, but it gets the results precisely the way Reg.exe Export produces them.) This screenshot shows the Capture-Recording.ps1 script running, with “The operation completed successfully” written every time Reg.exe was used.

When the script is done, it opens an Explorer folder window in the target directory. The next screenshot shows the mirrored file structure under the target directory and the RegMods.reg registration entries file. In this example, the script captured 1319 files in 76 directories and 1790 registry values in 1127 keys that were created or updated by the Seagate Crystal Reports 6.0 installation.

“Playing back” the App Install Recorder is as simple as copying the captured files onto the new system and running Reg.exe Import to import the captured registry data. When “playing back” a capture from a 32-bit system onto a 64-bit system, it’s important to ensure that 32-bit data ends up in the correct redirected locations. The AppInstallPlayback.cmd script shown in the next screenshot uses the 32-bit version of Reg.exe to import RegMods.reg so that keys are redirected to Wow6432Node as they would be for any 32-bit process. Similarly, it uses the 32-bit version of Xcopy.exe to copy the directory hierarchy, ensuring files are redirected to SysWOW64 when appropriate.

After creating an NTFS junction to “join” the installation directory to an equivalent name under “Program Files (x86)”, the Start Menu shortcuts shown here launched programs on 64-bit Windows that could not have been installed without Procmon.

AppInstallRecorderScripts.zip