Cool Whidbey debugger feature: Merge Environment

If your app depends on a particular set of environment variables, it could sometimes be a real pain to debug.

Case in point: The .NET profiling API depends on two environment variables to push the unmanaged profiling DLL into a process.  I covered this in my December 2001 column. When the CLR startup code executes, it checks for these environment variables, and if set correctly, loads the profiling DLL,

Debugging a profiler implementation (like my DNProfiler.dll) was a real pain. I couldn’t just launch the managed app under the IDE. Why not? Because the IDE didn’t use to give you any control over the environment passed to the debuggee. The target app’s environment was an inherited copy of the IDE’s environment.

You might be thinking “Big deal. Just set the environment variables you need at a command prompt before launching the IDE.”  There’s a catch: VS.NET itself uses the CLR.  Having my profiling environment variables set before launching the IDE would cause the profiling DLL to load up and affect the IDE. Not at all what I wanted.

To debug in this scenario, I had the following steps:

  • Compile an INT 3 into my profiling DLL’s startup code.

  • Start the IDE and a separate command prompt with the profiling environment variables set

  • Run my .NET app from this prompt

  • When my INT 3 got hit, the Just In Time debugging dialog comes up in response to the exception

  • Select the “debug in the IDE“ option, which caused my IDE to attach to my process

What a royal PITA!!!

In Whidbey, there’s much more control over the target’s environment. You can specify environment variables to be passed to the debuggee. You have two choices:

  • Pass your data as the complete environment to the child

  • Merge your environment variables with the environment that the IDE would normally pass on

This makes debugging my profiler DLL much simpler. In the Debugging options, I simply specify the additional profiling environment vars and select the “Merge Environment” option.


Comments (8)

  1. smidgeonsoft says:

    PEBrowse Interactive allows one to specify environment variables at process startup, but you must disable PI’s own default profiler as well in order to let your profiler into the application.

  2. Matt Pietrek says:

    I have to concur with Smidgeonsoft (e.g., Russ). PEBrowse Interactive has some fantastic debugging abilities that you won’t find in any other tool. If you’re a hardcore debugger geek, you’ve got to try it out. Right click on everything and see what you find.

  3. Fei Liu says:

    Have you guys heard of Ollydbg? 🙂

  4. Matt Pietrek says:

    Can’t say that I’d hear of Ollydbg. I just checked it out briefly, and it looks pretty cool. Sort of like PEBrowse Interactive.

    It’s good to see that there’s still a market for debuggers targeted at the hard core hackers. 🙂

  5. Fei Liu says:

    Since a lot of packers these days use anti-softice tricks to detect the existence of sice, ollydbg is a nice alternative to debug/study the schemes used in those kind of softwares.

    Ollydbg has a nice plugin that reset the TEB->PEB->isProcessBeingDebugged flag so it makes it harder for anti debugging software to detect its existence.

    Ollydbg is a ring3 debugger so in those occasions when you need debug device drivers etc, I guess the only options are sice or kd from MS. I really like kd with its powerful commands that easily reveal the kernel data structures.

    Btw, I have a question about sice. I am guessing when I bring out sice console window with ctl+d, there is only one thread running and that’s the sice thread in the system. Everything is under the control by sice when the console window is active, right?

  6. Da Feacha Autha says:

    This feature is in C++ or debugger only projects. Simply go to the project debugger settings pane to access the Environment section