Private Bytes Performance Counter -- Beware!
Q: When is a Private Byte not a Private Byte?
A: When it isn't resident.
The Private Bytes counter reports the commit charge of the process. That is to say, the amount of space that has been allocated in the swap file to hold the contents of the private memory in the event that it is swapped out. Note: I'm avoiding the word "reserved" because of possible confusion with virtual memory in the reserved state which is not committed.
So, if we are concerned with space allocated in the swap file then the Private Bytes counter is right on the money. However, that is not usually what we are concerned about. We're much more interested in memory pressure caused by copies of private bytes in multiple processes. That is to say we are concerned about the physical memory that has been allocated to hold those private bytes.
Why might/does the operating system allocate space in the swap file to hold the contents of memory whose contents have never become resident and may never become resident? The answer is not actually so complicated: Windows cannot deliver an "out of memory" exception/error just because you tried writing to, for instance, a static variable. The swap space must be pre-allocated at a reasonable time (such as loading a DLL) so that we can deliver an error result at a reasonable time -- the time at which the virtual addresses changed from reserved to committed.
So if we had some static data that could be modified, but usually isn't modified in practice, and maybe isn't even read, you would find that the Private Bytes counter overstated the cost of that storage. The only real cost is the small bit of housekeeping to track allocated space in the swap file. No private memory need be allocated.
In contrast, the output of vadump deals directly in what is resident and so, if memory is tight, it may understate the true memory cost because some things may have already been swapped out by the time you ask for the current resident pages. However, if memory is abundant then swapping is not going to happen and you can get a true picture of all the required pages from vadump.
And now a little data about our current product. Below is a table indicating the memory usage of the same managed notepad application Mark Russinovich wrote about last April. The Version 1.1 column shows the cost in kilobytes for the relevant class of memory in our Everett release and Version 2.0 shows the result in our current Whidbey release (actually this data is from a build that's a few weeks old now but it's probably still pretty close).
Summary
Version 1.1 Version 2.0 Delta
Kilobytes by Category Total Priv Total Priv Total Priv
Page Table 108 108 140 140 32 32
Other System 32 32 32 32 0 0
Code/Static 6008 1120 6520 568 512 -552
Heap 680 680 384 384 -296 -296
Stack 32 32 40 40 8 8
Teb 12 12 16 16 4 4
Mapped Data 488 0 356 0 -132 0
Other Data 644 640 408 404 -236 -236
Total Modules 6008 1120 6520 568 512 -552
Total Dynamic 1856 1364 1204 844 -652 -520
Total System 140 140 172 172 32 32
Grand Total 8004 2624 7896 1584 -108 -1040
Selected Modules
Version 1.1 Version 2.0 Delta
Kilobytes by Module Total Priv Total Priv Total Priv
sys.windows.forms.ni.dll 668 248 1280 80 +612 -168
mscorlib.ni.dll 640 320 992 88 +352 -232
system.drawing.ni.dll 176 80 244 32 +244 -48
system.ni.dll 164 84 276 32 +112 -52
mscorwks.dll 724 76 1084 60 +152 -36
mscorsn.dll 56 8 merged into mscorwks.dll
fusion.dll 152 12 merged into mscorwks.dll
(note that in v1.1 the native images didn't have .ni in the name
and there were two loaded modules for each ngen'd image)
Some interesting things to notice: even though the overall size of the .NET runtime modules went up in this test case, private pages are way down (1040k less) and overall memory usage is down because of data savings in Mapped data and Other Data -- meaning there is less general overhead.
So despite the fact that the libraries increased in size this application requires less memory in v2.0 -- and if you ran it alongside some other managed application you'd find yourself very pleased because of the low private bytes. Most of the memory would be shared so the second managed application would have much less impact on your system.
There are some great individual module results as well. Look at winforms -- even though the overall size of winforms went up, the private bytes fell from around 37% of the total to less than 7% of the total. The cost of the 2nd and subsequent winforms application has plummeted!
If in contrast, you look at the Private Bytes counter, you'll find that, as reported earlier, it's in the 7888k range. But I don't find that to be representative of memory pressure. That potentially private memory never makes an appearance.
The Private Bytes counter is definately overstating the true cost so take it with a grain of salt, just like everything else you read on my blog. :)