Using the "gu" debugger command to find the infinite loop


Somebody says, "Your program is consuming 100% CPU" and hands you a debug session. Usually, this happens because one thread has gotten stuck in an infinite loop. And if you're lucky it's the type of infinite loop that's easy to diagnose because it's just one function that isn't returning. (The more complicated types are where a function does some work and then returns, and then some of that work has a delayed effect that causes the function to run again, and so on.) Let's assume we're lucky because, well, debugging is an exercise in optimism.

The first step is to find the thread that is using all the CPU. That's actually pretty easy with the help of the !runaway debugger extension.

0:011> !runaway
 User Mode Time
 Thread    Time
 192c      0 days 0:05:22.457
 1384      0 days 0:00:16.063
 14ac      0 days 0:00:08.392
 48c       0 days 0:00:03.955
 1db0      0 days 0:00:00.010
 1888      0 days 0:00:00.010
 1078      0 days 0:00:00.000
 1470      0 days 0:00:00.000
 1f84      0 days 0:00:00.000
 1d60      0 days 0:00:00.000
 1850      0 days 0:00:00.000
 134c      0 days 0:00:00.000
 19fc      0 days 0:00:00.000
 1b4       0 days 0:00:00.000

Wow, thread 0x192c has sure used a lot of CPU time, but that doesn't mean that it's the thread that is in a 100% CPU loop, because the CPU time is cumulative over the lifetime of the thread. Maybe that thread has a lot of CPU time because it's been around the longest. What you need to do is resume execution for a little while, then break in again and see whose CPU time has increased.

0:011> g
^C
(1928.1d34): Break instruction exception - code 80000003 (first chance)
eax=7ffd9000 ebx=00000001 ecx=00000002 edx=00000003 esi=00000004 edi=00000005
eip=7c901230 esp=0124ffcc ebp=0124fff4 iopl=0         nv up ei pl zr na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=0038  gs=0000             efl=00000246
ntdll!DbgBreakPoint:
7c901230 cc               int     3
0:011> !runaway
 User Mode Time
 Thread    Time
 192c      0 days 0:05:23.679
 1384      0 days 0:00:16.063
 14ac      0 days 0:00:08.392
 48c       0 days 0:00:03.955
 1db0      0 days 0:00:00.010
 1888      0 days 0:00:00.010
 1078      0 days 0:00:00.000
 1470      0 days 0:00:00.000
 1ea4      0 days 0:00:00.000
 1d60      0 days 0:00:00.000
 1850      0 days 0:00:00.000
 134c      0 days 0:00:00.000
 19fc      0 days 0:00:00.000
 1b4       0 days 0:00:00.000

Aha, we see that thread 0x192c is the only one who gained any noticeable amount of CPU time. That's probably the one that's responsible for the 100% CPU usage.

0:011> ~~[192c]s
eax=00000000 ebx=77d5e581 ecx=0012daa0 edx=0000000c esi=01d18140 edi=00000000
eip=77d5e590 esp=0012da78 ebp=0012da88 iopl=0         nv up ei pl zr na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
USER32!FindWindowA+0xf:
77d5e590 50               push    eax
ChildEBP RetAddr
0012da88 1000714b USER32!FindWindowA+0xf
0012dbc8 100061f3 ABC!CAlpha::FindTarget+0x27f
0012dbf4 603517e8 ABC!CBeta::TransferData+0x18a
0012dc28 10002d9d DEF!CGamma:TransferData+0xc
0012dc48 00505303 ABC!CBeta::BeginAsync+0x51
0012dc5c 0090a21a GHI!CPrintSession::Open+0x2a51
0012dd20 009099d8 GHI!CPrintSession::Init+0x252
0012e060 009097e6 GHI!CPrintOptions::GetSettings+0x24a
0012e0a4 0090973d GHI!CPrintOptions::OpenSettings+0x248
0012e130 00909664 GHI!CDocumentMenu::OnInvoke+0x24
...

Now this is where the magical "gu" command comes in. You type "gu" to run the current function until it returns. If you get another prompt back, then type "gu" again, to run that function until it returns. And so on, until you find the function that doesn't return.

That's the one with the infinite loop.

Now you can start investigating why that function is stuck.

Comments (15)
  1. Mike Dimmick says:

    Process Explorer will show you CPU time per thread – just look at the Thread tab for the process which is consuming lots of CPU. If you have a recent version of the debugging engine and debugging symbols (or set up _NT_SYMBOL_PATH to point to a symbol server), you can then hit the Stack button to show the stack of that thread. You still need the debugging tools to use the ‘Go Up’ command, though.

    Get Process Explorer from http://www.microsoft.com/technet/sysinternals/ProcessesAndThreads/ProcessExplorer.mspx.

  2. dgt says:

    Mike,

    why use extra tools if one is already sufficient? why use calc.exe when you can issue "?2+2" in debugger command prompt?

  3. ThatsWhy says:

    > why use extra tools if one is already sufficient? why use calc.exe when you can issue "?2+2" in debugger command prompt?

    because Process Explorer can be used when debugging in Visual Studio instead of WinDbg/ntsd/others

  4. microbe says:

    I am confused. Maybe I am not a Windows expert, but under gdb you just attach to the process, switch to the thread context, ctrl-c to interrupt, and type “where” to see the stack.

    Why is it so complicated on Windows?

    [It’s just as easy on Windows. You use the tilde command to switch to the thread context and the k command to see the stack. But that’s now what this entry is about. -Raymond]
  5. greenlight says:

    Another way is to use Google Maps http://tinyurl.com/2psrqx

  6. Jeff says:

    Nice tip!  Thank you.

  7. JC says:

    Hi Raymond,

    Glad to see some great examples of what can be done with a lot of experience of the Windows Debugger like cdb etc…

    These tools really seem powerful and I’m trying to learn more about them these days: while they’re properly documented, sometimes the message gets more straight to the point using real life examples, especially because advanced debugging requires lots of experience and insight that can not be usually found in "dumb" documentation…

    Just wanted to let you know it was very interesting (to me, at least) and that I wouldn’t mind to see more entries like this (if you feel like writing them for your audience, of course)…

  8. other older new thing says:

    Somebody says, "Your program is consuming 100% CPU" and hands you a debug session.  Psychic debugging powers say "Because you only have one processor.  If you had two processors, you would have found the solution here:

    http://blogs.msdn.com/oldnewthing/archive/2005/12/02/499389.aspx

    "

  9. Norman Diamond says:

    Another way is to use Google Maps

    http://tinyurl.com/2psrqx

    Oh that’s hilarious.  Is that a real street?  Who assigned the name?

  10. Wolf Logan says:

    Is that a real street?  Who assigned the name?

    It’s the location of Apple’s headquarters in Cupertino, CA.

  11. Ben says:

    Is that really Apple’s headquarters?

    anybody else notice the Vista Dr just down the road? :)

  12. dsn says:

    Yes.  I always found it funny that Apple is at 1 Infinite Loop, while microsoft is at 1 Microsoft Way.  Wonder if that says anything about the respective corporate cultures?

    ;)

  13. Jonathan says:

    Is "gu" the same like hitting F11 in windbg?

    Being fairly experienced with the Windows Debugging Tools, I must say they are NOT well-documented. Their documentation is like man page – all the info is there, but not in a way mere mortals (=people who want to get their job done, not study the intricacies of these tools) can understand. They also lack in consistency – for example, there are several syntaxes to toggles (.enable_unicode 1, .lines+, etc)

  14. Woh, cool.  I did not even know that Windows had a command line debugger.  That (amongst other reasons) is one of my reasons for preferring to do development under Linux environments.  The Visual Studio graphical debugger is nice, until I need to get my hands dirty, then it gets in the way…

    I wonder if there is a Microsoft made equivalent to the Data Display Debugger.  DDD is the most ugly graphical debugger in the world, but insanely powerful because it can be used to graphically show data structures, unlike the VSS debugger, which just has this annoying flat layout of all my data (makes finding bugs in multi-dimension arrays a pain!).

    Though to be fair, the only reason I know about all the *nix command line based tools is because the graphical utilities tend to be so buggy as to be unusable (with the exception of DDD, which is just painful to use!).

  15. Here’s some more information on troubleshooting a High CPU Issue. I’m using a Test Scenario which means

Comments are closed.