You don't want to write an interop debugger.

  • Article

I've had a growing number of people inquire about how to write an interop-debugger with ICorDebug.  My goal here is to discourage you from doing that. (This reminds me of one of my college classes. On day one, the acting-Prof launched into a great sermon "Why you should drop this class now". It turned out to be a great class).

Here are some reasons that you should not try to write an interop debugger:

  1. It's a tough challenge, and Visual Studio already does it for you.
  2. You can actually get a lot of mileage out of managed only debugging. For example, you can use non-invasive native debugging APIs even when you're managed-only debugging. So you could use CreateToolHelp32Snapshot (or the managed wrappers in System.Diagnostics) to view native threads and modules. You can even load symbols for native modules and take mixed mode callstacks in a lot of scenarios.
  3. Some things are disabled when Interop-debugging, such as Edit-And-Continue. Interop-debugging is also not supported on Win9x, but more importantly it's not supported on 64-bit OSes, including Amd64 (although it will run in x64 in the Wow)
  4. Interop-debugging is very complicated, and consuming ICorDebug's interop-debugging interfaces are very difficult. Writing for Interop-debugging is basically the worst of all worlds. For example, you need to worry about Out-Of-Band events. Also, many operations that appear to be simple (such as step from managed code into native) are actually very complicated to implement and require the debugger to maintain separate managed and native debugging operations and then stitch them together to give the end-user an illusion of a unified debugging experience. The next few points are special case of this.
  5. There are a lot of threads.  You have multiple callback threads in the debuggee, non-callback threads (such as a UI thread) in the debuggee, and multiple threads in the debugger generating debug events.
  6. It causes ICorDebug to be reentrant, which results in some evil causality chains. For example, an innocent call to ICorDebugThread::EnumerateChains may block because the helper-thread hit an out-of-band event that needs to be continued by your ICorDebugUnmanagedCallback handler.
  7. You have more states to worry about. Managed-only debugging just has 2 states: Stopped (called Synchronized) + Running. Interop-debugging has more.

(Ok, so 4-7 are all under the same umbrella. That's because I really want to emphasize that it's complicated)