Many times in my life I have needed to debug a deadlock. You have one thread trying to acquire a critical section, and it can be a pain to determine which thread has it.
Setup: Go get OS Symbols. Having OS symbols is a must, and will make your life easier in many ways. To get OS symbols the easy way, use symbol server. Here is an article about how to do this — http://support.microsoft.com/default.aspx?scid=kb;en-us;319037. For Visual Studio.NET 2003 (and future versions), symsrv.dll is already included in the product. Here is another article on the subject if you prefer — http://www.codeproject.com/debug/postmortemdebug_standalone1.asp.
1. Go to the thread that is trying to acquire the critical section. Your stack should look like:
ntdll.dll!_ZwWaitForSingleObject@12() ntdll.dll!RtlpWaitForCriticalSection(_RTL_CRITICAL_SECTION * CriticalSection=0x00452ba0)
deadlock.exe!ThreadFunc(void * __formal=0x00000000) Line 24 + 0xd bytes C++
kernel32.dll!BaseThreadStart(unsigned long (void *)* lpStartAddress=0x00417681, void * lpParameter=0x00000000) Line 532 + 0x6 bytes C
2. Go to the ntdll.dll!_RtlEnterCriticalSection frame on the callstack
3. Open the memory window, set display to ‘4-byte integer’, and evaluate ‘@vframe’
4. The first DWORD should be the return address. In my example, this would be the address of ‘deadlock.exe!ThreadFunc Line 24 + 0xd bytes’. If this isn’t the case, then the debugger is having difficulty walking the stack. See if you can find the return address.
5. The DWORD after the return address is the address of the critical section. Drag this into the memory window
6. The fourth DWORD is the ID of the thread owning the critical section. You can drag this to the watch window if you want to see this value in decimal.
Alternatively, if @vframe is correct, you can evaluate this in the watch window ‘*((DWORD*)(*(DWORD*)(@vframe+4))+3)’. You still need to be at the ntdll.dll!_RtlEnterCriticalSection@4() frame:
*((DWORD*)(*(DWORD*)(@vframe+4))+3) 3304 unsigned long