Memory consumption issue with WinDbg, address summary


I must confess being pretty good at analyzing memory dumps from a performance or an exception perspective.  I have written a few articles and labs about those types of problems:

And one of my favorites here – “Must use, must know WinDbg commands, my most used”

Although I have a memory consumption lab here – “Lab 21: Debugging a W3WP process with high memory consumption”, I must also admit that these sometimes give me some trouble, I think mostly because I haven’t worked on so many memory consumption issues.  I would assume this is because I work primarily with Managed code and the memory is collected and protected by the garbage collection.

So any opportunity I get to look at a memory issue, I jump on it and like always want to share my perspective.

The first thing was to run the commands I always do, I run these commands regardless of what kind of memory dump or in what context the memory dump is taken in.  This helps me continue to build my perception of what ‘normal’ is.

Here are the commands I always run when I perform analysis of a memory dump.

  • !sos.threadpool
  • !mex.us
  • !mex.aspxpagesext
  • !mex.mthreads / !sos.threads
  • !mex.dae
  • !sos.finalizequeue

As this was a memory issue, I wanted and needed to run address –summary command.  After running this, I saw something I hadn’t seen…or maybe notices before.  The fact that the  Stack was consuming over 51% of the memory.

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                    114      7ff`a0b46000 (   7.999 Tb)           99.98%
Stack                                  2346        0`30c80000 ( 780.500 Mb)  51.19%    0.01%
<unknown>                               113        0`21670000 ( 534.438 Mb)  35.05%    0.01%
Image                                   372        0`0a44d000 ( 164.301 Mb)  10.78%    0.00%
Heap                                     72        0`02597000 (  37.590 Mb)   2.47%    0.00%
TEB                                     782        0`0061c000 (   6.109 Mb)   0.40%    0.00%
Other                                     8        0`001b9000 (   1.723 Mb)   0.11%    0.00%
PEB                                       1        0`00001000 (   4.000 kb)   0.00%    0.00%

That must be one heck of Stack.

A stack is simply a textual representation of what the thread is doing.  It contains the class, method and parameter list names which do consume memory, but if the stack is so big I would have expected a Stack Overflow Exception instead of what I saw above…  I wrote about a StackOverflowException here – “Capture a StackOverflowException and make a dump 0xc00000fd” .  And yes, there are certain types of variables stored on the stack which could consume memory, in contrast to the heap….I got that.  I also dumped the Native Stack Objects and the Dump Stack Objects and found nothing alarming.

But hey, it is what it is and I have confidence in the output of that command (address –summary), and I found the issue when I was executing !sos.threads, !mex.us and !mex.dae.

0:000> !sos.threads
ThreadCount:      4778
UnstartedThread:  0
BackgroundThread: 4777
PendingThread:    0
DeadThread:       0
Hosted Runtime:   no

That is a lot of threads and if the stacks are kind of long, then sure, they could be consuming some memory.

Running !mex.us, I saw that all of the threads has the same pattern, they were trying to Dispose an object which no longer existed and the stack up to that point and after that point was indeed pretty long, longer than what I would consider ‘normal’.

Dispose(Boolean)

Then, the output of !mex.dae showed +4000 exceptions.

In Generation: 2 from .NET v4.7.2053.00
HResult: 0x80131622
Type: System.ObjectDisposedException
Message: Cannot access a disposed object.
Stack Trace:

So, although the issue was raised as a memory issue, it actually turned out to be a coding issue that included many Monitor.Enter() methods, try…finally… code blocks and a Task.Result() call, and most of all the Monitor.Exit() method was within an if{} statement which, in this case was never true…

Comments (0)

Skip to main content