I notice one of the most common issues folks hit when they try to write their own managed debugger is forgetting to call ICorDebugAppomain::Attach() at the right time. I've seen this 3 different times recently. This part of ICorDebug is not documented very well, and so it's easy to understand the mistakes here.
You need to call ICorDebugAppDomain::Attach() from within the ICorDebugManagedCallback::CreateAppDomain callback and before you call Continue().
The consequence of not doing this includes:
- crashes, hangs, not getting notified of thread creations, and other random failures (beta2+ before).
- a DebuggerError callback with hr = CORDBG_E_APPDOMAIN_MISMATCH (post beta 2) shortly after continuing without calling Attach().
We actually added the runtime check to ensure Attach() was called and fire a DebugggerError callback sine this was such a common problem.
Attach() originally existed to support partial-process debugging scenarios (only debugging 1 appdomain) and multiple-users-per-debuggee scenarios (each users debugging their own appdomain). These scenarios have not actually been implemented in V2.0, but the APIs still exist for legacy purposes.
Another approach is to avoid writing against the ICorDebug APIs directly and instead write on top of an existing debugger (like MDbg) which takes care of these annoying details and tries to smooth out the API.