Every once in a while, I hear from a customer who has encountered an error while trying to install Document Explorer 2005 or Document Explorer 2008 (which is a prerequisite component that is installed during Visual Studio 2005 or Visual Studio 2008 setup).
How to diagnose this issue
In those cases, I typically ask the customer to send me the verbose installation log file from Document Explorer setup, which is located at %temp%\dd_dexplore*msi*.txt in order to narrow down the cause of this error. Then, I use the technique described in this blog post and search for the string "return value 3" to try to locate the action in the log file that indicates the root cause of the failure.
In many cases, Document Explorer setup failures end up with entries in the verbose MSI log file that look like the following:
12/01/07 11:11:11 DDSet_Status: BeginTransaction()->IHxRegisterSession::CreateTransaction() returned 8004036e.
12/01/07 11:11:11 DDSet_Error: BeginTransaction()->Attempt failed because another transaction was running.
More details about the cause of this issue
In most cases, this type of error means that a previous product installation was attempting to register a help collection, and something happened that caused that help collection registration to not complete correctly.
A customer recently posted a comment on one of my blog posts that describes this scenario in more detail. To summarize that comment, if another help transaction is running, a file named C:\Documents and Settings\All Users\Application Data\Microsoft Help\Rgstrtn.lck (or C:\ProgramData\Microsoft Help\Rgstrtn.lck on Windows Vista) will likely exist. If that file exists, it prevents other help collections from being registered on the system, and if a setup attempts to register a help collection during this time, it will fail with the type of error message listed above.
How to work around this issue
To work around this issue, you can reboot the computer (which will cause that file to be deleted in most cases), or you can manually delete the Rgstrtn.lck file to reset the state of the help collection installation system on your system. After getting rid of that file, you can attempt to install Document Explorer again.
Note - this type of help collection installation error can occur in any product that includes help collections as part of their installation logic. This error is not limited to Document Explorer setup.