Correlating the output of !eeheap -gc and !address


!address is a very powerful debugger command. It shows you exactly what your VM space looks like. If you already got the output from the !sos.eeheap -gc command (refer to this article for usage on sos), for example:


0:003> !eeheap -gc
Number of GC Heaps: 1
generation 0 starts at 0x01245078
generation 1 starts at 0x0124100c
generation 2 starts at 0x01241000
ephemeral segment allocation context: (0x0125a900, 0x0125b39c)
 segment    begin allocated     size
001908c0 793fe120  7941d8a8 0x0001f788(128904)
01240000 01241000  0125b39c 0x0001a39c(107420)
Large object heap starts at 0x02241000
 segment    begin allocated     size
02240000 02241000  02243250 0x00002250(8784)
Total Size   0x3bd74(245108)
——————————
GC Heap Size   0x3bd74(245108)


you can correlate the segments with the output of !address to get a better view of them. For this specific case here’s the excerpt of the output from the !address command:


0:003> !address
[omitted]
    01232000 : 01232000 – 0000e000
                    Type     00000000
                    Protect  00000001 PAGE_NOACCESS
                    State    00010000 MEM_FREE
                    Usage    RegionUsageFree
    01240000 :
0124000000052000
                    Type     00020000 MEM_PRIVATE
                    Protect  00000004 PAGE_READWRITE
                    State    00001000
MEM_COMMIT
                    Usage   
RegionUsageIsVAD
               0129200000fae000
                    Type     00020000 MEM_PRIVATE
                    Protect  00000000
                    State    00002000
MEM_RESERVE
                    Usage   
RegionUsageIsVAD
              
0224000000012000
                    Type     00020000 MEM_PRIVATE
                    Protect  00000004 PAGE_READWRITE
                    State    00001000
MEM_COMMIT
                    Usage   
RegionUsageIsVAD
              
0225200000fee000
                    Type     00020000 MEM_PRIVATE
                    Protect  00000000
                    State    00002000
MEM_RESERVE
                    Usage   
RegionUsageIsVAD
    03240000 : 03240000 – 73050000
                    Type     00000000
                    Protect  00000001 PAGE_NOACCESS
                    State    00010000 MEM_FREE
                    Usage    RegionUsageFree
    76290000 : 76290000 – 00001000
                    Type     01000000 MEM_IMAGE
                    Protect  00000002 PAGE_READONLY
                    State    00001000 MEM_COMMIT
                    Usage    RegionUsageImage
                    FullPath C:\WINDOWS\system32\IMM32.DLL
               76291000 – 00015000
                    Type     01000000 MEM_IMAGE
                    Protect  00000020 PAGE_EXECUTE_READ
                    State    00001000 MEM_COMMIT
                    Usage    RegionUsageImage
                    FullPath C:\WINDOWS\system32\IMM32.DLL
[omitted]

——————– Usage SUMMARY ————————–
[omitted]


——————– State SUMMARY ————————–
    TotSize   Pct(Tots) Usage
   0275c000 : 1.92%      : MEM_COMMIT
   7b20a000 : 96.20%      : MEM_FREE
   0268a000 : 1.88%      : MEM_RESERVE


Largest free region: Base 03240000 – Size 73050000


This says that the 2 segments (starting from 01240000 and 02240000) are adjacent to each other – part of them are committed, the rest is still reserved memory. Before and after the 2 segments we got some free space there. As I mentioned below it’s very unlikely that the managed heap is fragmenting the VM because we are good about requesting large chunks at a time and usually the OS is not bad at giving us addresses that are pretty contiguous if possible. One of the very few cases where you would see managed heap fragmenting VM is if you have temporary large object segments and GC needs to frequently acquire and release VM chunks. Those chunks could be scattered in the VM space especially considering there are other things that consume VM as well at the same time.

Comments (15)

  1. nativecpp says:

    Couple of questions:

    1) From your eeheap dump, I can see that gen2 start @ 0x01241000 which is shown in ephemeral segment listing. How about the other segment 0x793fe120? What are they and can you explain ?Because 0x1241000 is gen2, GC will eventually compact even though it is expensive, right ? you said that "you have temporary large object segments", wouldn’t that be in LOH @ 02241000?

    2) When I did a dump in my VS2005, I got the following:

    Number of GC Heaps: 1

    generation 0 starts at 0x014aeff8

    generation 1 starts at 0x01450c38

    generation 2 starts at 0x01411000

    ephemeral segment allocation context: none

    segment    begin allocated     size

    001bf238 7a721784  7a74248c 0x00020d08(134408)

    001bece8 7b451688  7b467f9c 0x00016914(92436)

    001960c8 790d6358  790f5800 0x0001f4a8(128168)

    01410000 01411000  015038f0 0x000f28f0(993520)

    Large object heap starts at 0x02411000

    segment    begin allocated     size

    02410000 02411000  02417fd8 0x00006fd8(28632)

    Total Size  0x15038c(1377164)

    ——————————

    GC Heap Size  0x15038c(1377164)

    Why do I have more segments than your ? Can you explain the segments in my dump ?? Is it useful to look at those segment ?

    Thanks

  2. maoni says:

    The first segment in SOH (Small Object Heap)

    001908c0 793fe120  7941d8a8 0x0001f788(128904)

    is a read only segment for frozen objects. http://msdn2.microsoft.com/en-us/library/system.runtime.compilerservices.stringfreezingattribute_members.aspx has an explanation for frozen objects. Those objects are always considered live.

    Yes the LOH heap is, in this case, the segment starts from 02240000. And yes if you do a full colletion (meaning a gen2 collection) it will compact the segment starts from 01240000 and form free lists in the 02240000 segment (since we don’t compact LOH segments).

    When I said one of the scenarios that could cause VM fragmentation is temporary large objects, I meant if you have temporary large objects you will likely need to release a LOH segment (when the large objects on that segment are all dead) and re-acquire a new one from the OS (when you create new large objects). This can cause VM fragmentation.

    In the !eeheap -gc output you showed, the first 3 segments are read only segments. You can tell because the address for "begin" is very different from the address for "segment". You should look at the segments if you have a reason to, for example, if you think they consume too much space in which case you’ll want to follow the suggestions I gave in the MSDN article to investigate.

  3. nativecpp says:

    Thanks for the clarification. So, are those read-only segments belong to gen2 ? I am curious how come the GC has those segments Since I didn’t use ngen or use StringFreezingAttribute?

  4. maoni says:

    Well, those segments are not exactly part of any generation – if you have to clarify you can say they are part of gen2. There are already frozen strings declared by the .NET framework assemblies that you use, which is why you have those even though you didn’t declare any in your own code.

  5. nativecpp says:

    Hi Maoni,

    I have one more question πŸ™‚

    I thought with demotion and segment reuse in .Net 2.0,

    memory problem(s) (fragmentation, OOM)in gen0-gen2 will be less of an issue. So, my thinking is that I should concentrate in LOH for framentation which starts @02241000 in your example. In your first entry, you indicated the

    "This says that the 2 segments (starting from 01240000 and 02240000) are adjacent to each other – part of them are committed", shouldn’t we look at address >= @02241000 ? Am I missing something ? BTW, we all know that LOH start @02241000, where does it end ? Do we look at memory with usage is RegionUsageIsVAD ?

    Thanks

  6. maoni says:

    Hi nativecpp,

    First of all, do you have a memory usage problem? If not this shouldn’t be the area that you concentrate on. You are right that after the work in 2.0 fragmentation in SOH should be much less of a problem. Fragmentation in LOH can be used for LOH allocation so it’s in general not a concern.

    HTH.

  7. nativecpp says:

    Fortunately, I don’t have a memory usage. But I do like to know more about the !eeheap -gc and !address

    so that if and when I do have a memory issue, I can go right into the problem because I am sure when it happens, I need to fix it fast πŸ™

    Sorry if I sounded like a nagger to you as I wanted to know every detail of GC topic you brought. I just would like to know more and more about to ‘satisfy’ my own "inquery mind".

  8. maoni says:

    No problem. Curious minds are always welcome. As I said fragmentation in LOH is generally not a concern because it’s reused for allocations.

    So to answer your previous questions about the LOH segment –

    The !eeheap -gc indicates:

    Large object heap starts at 0x02241000

    segment    begin allocated     size

    02240000 02241000  02243250 0x00002250(8784)

    and !address shows the detail of this segment:

    02240000 – 00012000

       Type     00020000 MEM_PRIVATE

       Protect  00000004 PAGE_READWRITE

       State    00001000 MEM_COMMIT

       Usage    RegionUsageIsVAD

    02252000 – 00fee000

       Type     00020000 MEM_PRIVATE

       Protect  00000000

       State    00002000 MEM_RESERVE

       Usage    RegionUsageIsVAD

    So the committed part ends at 02240000 + 00012000 and the reserved part ends at 02252000 + 00fee000.

    RegionUsageIsVAD includes all the explicit VirtualAlloc’s + other things. Since GC heaps are allocated with VirtualAlloc’s you will always see RegionUsageIsVAD as the Usage for GC heap segments.

  9. Keith Hill says:

    I’ve been playing with SOS, namely !dumpheap -type to get addresses of objects. I was under the impression that when an object survived a gen 0 collection and was moved to gen 1 that it was copied to a different region of memory.  I am not observing this (well not always) when I examine the addresses of some large arrays (45K in size).  Although if I do generate a bunch of other objects and do a GC.Collect() then I do notice that the arrays are moved in memory.  What determines when objects are moved?  Is it just compaction or is it always when moving from gen 1 to say gen 2?

    I’m curious because we have a .NET 1.1 class that holds onto arrays.  If the array is larger than 64K then we segment the array using a jagged array into multiple 64K segments to avoid using the LOH (which never compacts). I’ve argued that this isn’t a good idea due to excessive copying during compaction and during promotion in the SMO.  However my experiments didn’t back my hypothesis WRT copying during promotion.  Am I wrong on this point?  BTW any other downsides that you can think of WRT segmenting large arrays like this?  Thanks!

  10. maoni says:

    Hi Keith, I am curious to know why you decide to not use LOH. If it’s because the large objects are temporary, you could consider having a pool so you can reuse them.

    When something is in the SOH you should assume that it *can* be moved unless it’s pinned.

  11. Keith Hill says:

    We have some applications that crunch on very large arrays (250,000 up to 1 million element arrays) of ints and doubles.  The arrays come and go fairly often (data acquisition) and was causing OOMs which the team was pretty confident were a result of severe fragmentation of the LOH.  

    I’m worried though that allocating such large chunks of memory on the SOH (say 16 up to 64 64KB chunks), even though it tends to be allocated contigously, can also cause perf problems.  Unless the SOH algorithm is smart enough to keep these jagged-ized arrays close together you could get poor performance when indexing through them due to poor sequential locality.

    Funny you should mention it but I have suggested that we create a LargeArrayPool.  One minor problem is that managed arrays are fixed size so we would pre-allocate the pool arrays based on a max capacity and then the user of the array would have to access the array’s actual length via the pool manager or some other object.

  12. maoni says:

    I would first confirm that the OOM was due to the fragmentation of the LOH.

    Yes the SOH algorithm would naturally keep arrays allocated together stay together during compaction.

  13. Vish says:

    Hi Maoni,

    I have multiple LOH segments in the same heap. Why are multiple LOH segments created? Doesnt GC allocate LOH segments in blocks of 64 MB? None of the segments are 64 MB. The largest free block in the first LOH segment is 5.6 MB and none of the objects in the second LOH segment is larger than 5.6 MB.

    Heap 0 (000eb2e0)

    generation 0 starts at 0x11f2944c

    generation 1 starts at 0x11db9ac4

    generation 2 starts at 0x102d0038

    ephemeral segment allocation context: none

    segment    begin allocated     size

    30bee928 7b451688  7b467f9c 0x00016914(92436)

    00100458 7a721784  7a74248c 0x00020d08(134408)

    000f47e0 790d6358  790f5800 0x0001f4a8(128168)

    102d0000 102d0038  1274c040 0x0247c008(38256648)

    Large object heap starts at 0x202d0038

    segment    begin allocated     size

    202d0000 202d0038  220f9040 0x01e29008(31625224)

    471f0000 471f0038  482ceed0 0x010dee98(17690264)

    7d0e0000 7d0e0038  7db7e9e8 0x00a9e9b0(11135408)

    Thanks!

  14. maoni says:

    Vish,

    >>>Doesnt GC allocate LOH segments in blocks of 64 MB

    No. And you shouldn’t rely on any specific segment size ’cause it may change with each release.

    >>>The largest free block in the first LOH segment is 5.6 MB and none of the objects in the second LOH segment is larger than 5.6 MB.

    LOH is not compacted.

  15. Kumar says:

    Hi Maoni

    I’ve a question as regards deciphering the boundaries for a particular GC segment.

    For eg. given the details below

    ————————————————————–

    02240000 – 00012000

      Type     00020000 MEM_PRIVATE

      Protect  00000004 PAGE_READWRITE

      State    00001000 MEM_COMMIT

      Usage    RegionUsageIsVAD

    02252000 – 00fee000

      Type     00020000 MEM_PRIVATE

      Protect  00000000

      State    00002000 MEM_RESERVE

      Usage    RegionUsageIsVAD

    ————————————————————-

    how do we know that the address regions starting at 02240000 & 02252000 addresses belong to the SAME GC segment?

    Becuase !eeheap -gc tells me only about the committed bytes in a segment and does not talk about the reserved bytes for the segment?

    Please clarify if I’m missing something here

    Many thanks for enlightening me about GC