Before you can learn to recognize what’s wrong, you must learn to recognize what’s right


Sometimes, when I'm debugging a problem, I'll ignore a particular thread and somebody will ask, "What is that thread doing, and how did you know to ignore it?"

My reply: "I have no idea what it's doing, but whatever it's doing, it's normal."

Tess has been running an excellent series of posts on debugging the CLR, but one of the most important lessons is where you learn about things to ignore when debugging an ASP.NET hang.

Hangs and deadlocks are tricky to debug because there is no unhandled exception that says, "Look at me, I'm a bug!" Instead, the program just grinds to a halt and you have to go spelunking to figure out why. What you are looking for is anything out of the ordinary, but in order to recognize what that is, you first need to know what is ordinary.

So do that. Run your program, let it do its thing, then break in with the debugger and take a look around. Make a note of what you see. Those are things that are going on when nothing is wrong. This is what your program looks like when it is running normally. Now that you know what normal operations look like, you can recognize the abnormal stuff.

Note that you don't even have to know what all those normal things are. For example, when I connect to a process with the debugger, I often find threads lying around which are waiting inside RPC or the kernel thread pool. I don't know what they are doing, but since they are always there, I don't pay much attention to them.

Comments (11)
  1. dave says:

    Ah, the untransferability of knowledge.

    "What are you looking for with the debugger?"

    "No idea. Just for stuff that isn’t right."

  2. Mike says:

    A&B: (looking at a hex dump of some kernel-mode memory)

    A: Lets see where this pointer goes.

    B: How do you know it’s a pointer (as opposed to any other random bunch of bits)?

    A: It looks like one.

    Happens to me all the time.

  3. mikeb says:

    Speaking of RPCs…  there are times when I’m looking at the server side of an RPC call and would like to know what the situation is on the client side (to keep things simple, let’s say both are on the same machine).  

    I’m thinking there has to be a relatively easy way to find the client-side stack trace, but the best answer I’ve found so far is the rcpexts.dll extensions for ntsd/cdb or the closely related DbgRcp tool.  These tools are by no means simple and straight forward, and I feel like I’m missing some obvious simple technique.

    Does anyone know a simple technique to dump the stack for the thread on the other side of an RCP call?

  4. Dustin Long says:

    "There’s way too much information to decode. You get used to it. I don’t even see the code.  All I see is Blonde, Brunette, Redhead…."

  5. Grzegorz says:

    I need to contact this guy (oldnewthing). He has nothing in his profile, he has no bio. Does he have some email address?

  6. Cooney says:

    most likely yes.

  7. Dean Harding says:

    Grzegorz: There is a "Contact Me" link on the right which points to: http://blogs.msdn.com/oldnewthing/articles/232903.aspx

  8. steveshe says:

    Before you waste your time trying to get Raymond to do something for you, you might want to read this: http://blogs.msdn.com/oldnewthing/articles/166545.aspx

  9. Grzegorz says:

    Thanks Dean, it’s hidden so well I wouldn’t spot it :). Steveshe, why you falsely assume I’m going to try get him to do something for me?

    And sorry for the off-topic comment, but aricle I have a question about is old and comments there are disabled.

  10. See what each thread is waiting for.

  11. Michael Mosley tests a theory by Richard Wiseman.

Comments are closed.