Cool MSI custom action debugging tool

I posted an item on my blog last week with some details about how Windows Installer custom actions work behind the scenes. Some of the feedback I got back after posting this item asked some good questions about debugging strategies for custom actions. Based on that, I wanted to share a wrapper DLL written by Jungwook Bae, a developer on the Visual Studio and .NET Framework setup team. His tool makes the process of implementing and debugging DLL custom actions much simpler. You can download the tool itself by clicking here. The following is more information about the wrapper DLL and how to use it:

CATest - a tool that will help developing custom action DLLs

One of the biggest pains when writing custom actions is whenever you modify the DLL, you have to stream the new DLL into the MSI and restart setup. Deferred custom actions are the most difficult because you have to wait until all the files are copied and registry values are written again, which can take a very long time for large products.

To get around this problem, you can use catest.dll as a wrapper for the custom action DLL you are writing. You can test your custom action DLL without streaming it into your MSI, and you can run it as many times you want without restarting setup.

How to use CATest:

1. Add catest.dll to the binary table of your MSI (by using Orca or some other tool)
2. Share out your build folder so that your DLL and PDBs are accessible from a test machine
3. Modify catest.ini to match your custom action name, path to DLL and EntryPoint. You can debug the following 4 types of custom actions: Immediate, Deferred, Rollback, and Commit. You do not need to include all 4 sections in catest.ini; instead you can include only the ones you want to test.
4. Copy modified catest.ini to %windir% on the machine we are testing.
5. Modify the custom action records in the CustomAction table of your MSI to point to catest for each of the custom actions you want to debug. Each custom action you want to be able to debug should have "catest" as the Source value and the Target value, and should match the custom actions that you list in catest.ini.
6. Now you are ready to go, so run the setup. When prompted to attach a debugger, you can attach your favorite debugger and click OK. Then, you can load your symbols and open source files to step through the code. When the custom action finishes, a dialog will ask if you want to retry the custom action. You can update your code and rebuild the DLL, then choose to retry the custom action and debug it again within the same installation session. Note: you will have to reload your symbols if you rebuild and rerun the custom action.

<update date="3/24/2009"> Fixed broken link to download the debugging tool. </update>

• Anonymous
  March 07, 2005
  Aaron - Kudos to you and Jungwook for sharing this with the community.
  
  Continuing the discussion from the other thread on debugging... To hit the rollback case for debugging you can return an ERROR_INSTALL_FAILURE from another deferred CA (added after the deferred CA you want to debug, of course). All of my CA DLL's have an entry point MsiCauseError() for this purpose.
  
  The QA people tend to frown when they grab a work in progress and see the "Do you want me to cause an error?" dialog. I haven't gotten the "Installation rollback after dialog selection" bug report yet, but if any of them happen to be reading this, I'm sure I'll see one assigned to me in the morning :)

• Anonymous
  November 13, 2008
  Aaron, I second Steven's praise.  However, while using catest to work on WiX v. 3's custom actions, I was wondering if it is possible within the catest.ini to have hooks for multiple actions of the same type (e.g. Immediate); I figured that catest could route to the correct action by the CustomAction.Action field.  Here's my CustomAction table, to illustrate my point: SchedSecureObjects 1 WixCA SchedSecureObjects SchedSecureObjectsRollback 1 WixCA SchedSecureObjectsRollback ExecSecureObjects 3073 WixCA ExecSecureObjects ExecSecureObjectsRollback 3329 WixCA ExecSecureObjectsRollback SchedServiceConfig 1 WixCA SchedServiceConfig ExecServiceConfig 3073 WixCA ExecServiceConfig RollbackServiceConfig 3329 WixCA RollbackServiceConfig I have no problems setting catest up for one of each type, but what would I do in the INI in order to debug both SchedSecureObjects and SchedSecureObjectsRollback? If catest doesn't handle this situation, is it possible to get source so that I can modify it to do so?  Otherwise, I'm stuck reverse-engineering it (or writing a dll with yet another layer of indirection... hmm... that would probably work). Thanks, Tom

• Anonymous
  November 14, 2008
  Hi Tomtrias - I will follow up with the creator of this CATest tool to see if it is possible to send you source code.

• Anonymous
  March 17, 2009
  The is an easier way to debug the custom actions. Simply enter the following in the custom dll: #if DEBUG   Debugger.Launch(); #endif Then attach to the VS instance that has the code.

• Anonymous
  September 04, 2010
  @mendezam well i think you dont get the idear. The cool stuff is not tha fact to use a debugger. Its tha fact that you can rebuild and restart your CA without the need to restart your setup. So you can call it edit & continue ca debugging.

• Anonymous
  April 01, 2013
  hi, I believe that this is really helpful to a lot of ppl. I am not able to understand How to use CATest, it would be very nice if you can share some sample project with uses CAtest to debug

• Anonymous
  April 01, 2013
  Hi Onam - I'm sorry, but I don't have any sample projects.  There is a readme.txt inside of the .zip file for this tool that provides more details that might help get you started.  Can you please take a look at that and let me know if you have any specific places where you get stuck?

• Anonymous
  May 08, 2013
  The description of this is hard to understand.  It would help if you give an example of just perform a deferred mode call to the CustomAction, what are the steps you need to go through to set that up.   Thanks!