Debugging Series: Symbols Server and Your Symbols

Last post, I gave you a simple introduction to Symbol Servers. I showed you how you can use the options in Visual Studio to quickly get access to Microsoft public symbols and how that could give you more information for your debugging sessions. However, I did not show you how you could add your owns symbols to the server, and more importantly, why that is a great thing to do.

Adding Your Symbols

Adding your symbols to the symbol server is a great practice, as it gives you a better chance of debugging one of your own executables or dlls without having to have that exact version of the image available. The typical use case here of course is support of a previously deployed app. You’re currently heads down working on V3 of your app but some customer out there is currently using V1 and reports some crashing or other bug with that version. How to maximize your chances of debugging the issue? Symbols server will certainly help, as long as you have added your PDBs to the server.

NOTE:

This is all predicated on the fact that you have indeed saved your PDBs. Even if you don’t plan to setup a symbol server, please please please save your PDBs!

How do you add your own symbols to the server? There’s an “easy” way, and then a “hard” way.

The Easy Way

The “easy” way is to use the support built into Team Foundation Server 2010’s Team Build. All you need to do is tell TFS where to drop your PDBs ( it must have permission to write to that directory ) and it will handle all the rest! You do that in your build definition file. Just set the “Path to Publish Symbols” to a public UNC share, and your golden. Here’s an example:

image

Behind the scenes, TFS is actually handling the steps you can manually do, which I’ll describe now.

NOTE:

I’ll be discussing the details of what the Source Server is all about in a future post. Stay tuned.

The Hard Way

You can manually add your PDBs to a symbol server by making use of the symstore.exe, which is part of the “Debugging Tools for Windows” package. Of course, since this is just a command line tool, you can incorporate the following steps into any MSBuild process.

Here’s an example of the command line options that I used to push my symbols to my store:

symstore add /f DebuggingSeries.* /s \\camerons4\Symbols\MySymbols /t "My Version 1" /v "1.0.0.0" /c "Manually adding"

This simply takes the exe and the PDB from my output directory ( the directory I ran symstore in ), and copied the symbols to the UNC folder specified.

Check out this page in the MSDN docs for more details, and here for all the details around the command line options.

If you take a look at the folder where the PDBs and exe get pushed, it should look something like this:

image

But what does this now do for me? Easiest to show through an example.

Symbols In Action

As I mentioned earlier, the classic case where use of a symbol server can really help is during a post-mortem debug session, where you have received a mini-dump of your application. So let’s simulate that use case by running the following bit of code on one machine, produce a mini-dump of that running exe, then take that dump over to another machine that does *not* have the source to the exe, load it into VS 2010, and see what we see.

  1. Here’s the code I’m using for this example:
    image
    This code will run and break on the 50th iteration, allowing you to get a mini-dump in a later step.
  2. Build that in Release mode, choosing “Any CPU” as the configuration.
  3. Add the new symbols from your release directory to your build server, using the symstore.exe command I described above.
  4. Run the app by hitting Ctrl-F5 in the Visual Studio. This will run your app without launching the debugger.
  5. If you’re using the code above, you’ll see a console window appear and then an apparent “crash” dialog:
    image
  6. At this point, bring up the Task Manager by hitting Ctrl-Shift-Esc. Find your app in the process list, right click, and select “Create Dump File”:
    SNAGHTML3aae9d4
    A dialog will appear after a second telling you the location and name of the dump file that was just created. Keep that dialog up and handy, as you’ll be copying that dump file over to another machine next.
  7. Copy the dump file over to another machine.
  8. Open up Visual Studio 2010 and open the dump file.
  9. Ensure that you have enabled all the symbol server settings in Visual Studio as described in my last post, as well as turned off “Just My Code”.
  10. Add a new entry into the list of Symbol file locations that points at the UNC share that you enabled for your application symbols. My dialog looks like this:
    image
    NOTE:
    There is a bug when attempting to open up a mini-dump for managed code in VS 2010 SP1. The workaround is to make a couple dlls available from the machine that the dump was generated to the machine that wants to analyse a dump. Details here. That’s what the last entry in the above image is doing, making those dlls available.
  11. Now open the dump file in VS. You should see something like this in the VS doc well:
    image
  12. Click the “Debug with Mixed” link, outlined in read in the above image.
    ( I think mixed-mode debugging is worthy of its own post, so I’m not going to go into details on mixed-mode debugging right now.  )
  13. Visual Studio will go into a debug-like state. After a couple moments, take a look at your call stack and “Locals” windows. You should see your code showing up in the stack, as well as the value of your variables in the Locals window. Mine looks like this:
    SNAGHTML3c2d146
    And my Locals like this:
    image
  14. Very cool! Now if you didn’t turn off “Just My Code” and were not using a symbol server, you would be presented with this dialog after clicking the “Debug with Mixed” link:
    image
    After hitting the “OK” button on that, your call stack would look something like this:
    image
    Not real useful.

Next Steps

I hope that gives you a good sense of how proper use of symbol servers can help you. In my next post, I’ll talk about a sister technology to symbols servers: Source Server. That will give you the added benefit of automatically opening the actual source code in the above scenario, in addition to the added data in the call stacks.

Stay tuned!

Cameron