Remote kernel or user mode debugging of dumps or live systems

  • Article

 

GES (Global Escalation Services) is not only responsible for helping our external customers, but we spend a great deal of time collaborating with engineers and developers around the world at our support and development sites. We often look at large dump files, but in some cases we perform a live debug to determine root cause of a server failure. In the case of a memory dump, the files are usually very large, so copying the files over the network, even on the fastest WAN connection, can take a LONG time.

The solution is remote debugging.

Here is how you do it!

First of all it takes two to tango: a remote person and the expert who will help the remote person by debugging the process using the debugger installed on their machine.

Let’s say you are the expert who is helping the remote person. Here’s how the process works:

1. The remote person opens a dump file, debugs a process, or kernel debugs a machine at some remote location using windbg.

2. The remote person decides, “I NEED HELP!”

3. The remote person simply types in .server tcp:port=9999 at the windbg prompt.

 

Notice the following output.

Server started. Client can connect with any of these command lines

0: <debugger> -remote tcp:Port=9999,Server=MyServerName

 

1. The remote person sends email or IM to the person they want help from with the connection string <debugger> -remote tcp:port=9999,Server=MyServerName

2. The expert runs WINDBG -remote tcp:port=9999,Server=MyServerName from the debuggers directory.

3. At this point the remote person should see the following message at the remote debugger site.

 

EXPERTMACHINE\expert (tcp 165.33.5.122:54546) connected at Tue Mar 25 15:36:53 2008

 

Once connected, the expert can issue any command to debug the dump or target machine remotely. The great part is many people can connect to the remote debugger session if needed. It’s a great collaboration tool, and something we use every day at Microsoft.

 

When remote debugging, I find it useful to save the debug session in a log file. It’s as easy as typing .logopen C:\mydebuggersession.log in windbg before the remote debug session starts. This gives everyone the opportunity to look at the debug session later if necessary.

 

One last thing to keep in mind about remote debugging is security. I recommend using the .noshell command to prevent the execution of remote shell commands. Without the .noshell command, it is possible for people connected to your session to use the .shell (Command Shell) command to execute an application or a Microsoft MS-DOS command directly from the debugger.

 

Thanks Jeff-