Hardmode vs. Softmode

User-mode debugging can be split into 2 models: Hardmode and Softmode.

• Hardmode means using the native debugging APIs. This means all threads are stopped by the OS at each debug event and there’s no helper-thread. The debugger does not need to run any code in the debuggee’s process.
• Softmode means not being hardmode, and thus we use our own mechanism for communicating debug events and stopping the debuggee. This also means the debuggee has an in-process piece that provides information to the debugger and services debugger requests. Softmode does not necessarily mean the inprocess piece has a helper-thread.

The CLR's managed debugging is softmode, and has a helper-thread and its own IPC protocol to send managed debug events.

Each of these has some advantages over the other (which I list here), but there are some things that can only be done in one mode or the other.

Things that can only be done in Hardmode:

- The only truly robust way to debug the native portion of the debuggee. Interop-debugging’s fragility comes from the soft-mode nature of managed-debugging.

- Allows dump-debugging. Softmode can’t do dump debugging because you can’t run any code in the debuggee.

Things that can only be done in Softmode:

- Multiple debuggers attached to same debuggee. Windows only allows 1 debugger per debuggee. Since softmode is not actually OS-debugging, it can avoid this restriction, though our current managed debugging services does not yet support it. It’s true that you could have a single debugger attached then act as a multiplexer to service multiple debuggers, but then that would require an additional protocol between the debuggers and the mux. Since there’s no such standard for that protocol, the mux and all debuggers attached to it would have to be from the same vendor.

- Debuggee insulation against debugger crashing. With native-debugging, if the debugger crashes, the OS will automatically kill the debuggee. This means that it’s dangerous to debug a highly-reliable app (such as a production server), even if you’re only doing “safe” operations.

- Debuggee security from debugger. A native debugger can call APIs like WriteProcessMemory and SetThreadContext so it can definitely completely control the debuggee. This can be a problem if the debuggee has sensitive data you don’t want a debugger seeing. With softmode, you could have a debugging architecture where the debugger has no direct access to the debuggee process and must go through a well-defined interface (eg, perhaps DCOM). This gives the inprocess piece in the debuggee a chance to validate all activity from the debugger. Thus a debuggee could be safe even in the face of a malicious debugger client.

- [Updated] Partial Process debugging. Currently on Windows, a native debugger debugs the entire process at once. Sometimes it's desirable to just debug a subset of the processes (perhaps a given thread or a given appdomain). For example, perhaps you have a large gui application (like a video game) that just hosts a scripting engine and you want to just debug your scripts without actually debugging the whole app. Since softmode debugging controls the debuggee suspension, it can work around these limitations. Hardmode's very definition implies that you're debugging the whole process.

• Anonymous
  February 09, 2005
  The comment has been removed
• Anonymous
  February 11, 2005
  Pavel - great point, thanks for bringing that up.
  The MSDN link for DebugSetProcessKillOnExit is at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/debug/base/debugsetprocesskillonexit.asp
  (The API is new in XP/Win2k03)
  
  Another random issue with the rude-detach would be if any threads had an outstanding single-step flag set (those this is a much smaller window than outstanding BPs or suspended threads). That exception would likely go unhandled and then tear the debuggee down.
• Anonymous
  February 14, 2005
  There is a subtlety in how native debuggers work that you missed. Although only one process can listen on a debug port, the dbgeng.dll (part of NTSD/WinDBG/KD) acts as a multiplexer, allowing multiple native debuggers and/or tools to connect and debug a single process.
• Anonymous
  February 14, 2005
  Barry - I did mention multiplexing above:
  "It’s true that you could have a single debugger attached then act as a multiplexer to service multiple debuggers ...".
  The public native-debugging API, kernel32!WaitForDebugEvent, doesn't expose multiplexing.
  I also thought that Windbg allowed multiple debuggers to inspect a debuggee, but I didn't think it allowed multiple debuggers to control a single debuggee.
• Anonymous
  October 31, 2008
  In my previous post I mentioned that CLR 4.0 will support managed dump debugging through ICorDebug, and