High Memory part 4 - Managed Strings

  • Article

So the last high memory post, High Memory part 3, focused on native memory that was the problem, let's get back to our normal, managed world.  So other then DataTables which we save in the previous post, High Memory continued, what else could cause us to see a large amount of memory being used by .NET?

There are a bunch of other problems that can cause high memory.  If most of the memory is being taken up by .NET, then we can look at the output of !sos.dumpheap -stat to see what is taking up most of the memory.  One common thing to see is something like:

 

 0:000> !clr10\sos.dumpheap -stat
...
Statistics:
        MT      Count     TotalSize Class Name
...
0x0b80a234     11,685       233,700 System.Web.FileMonitorTarget
0x0d81f344      6,044       241,760 System.Web.UI.WebControls.Style
0x79ba968c      5,948       309,296 System.Collections.Hashtable
0x0f4fa25c     22,641       362,256 System.Web.Configuration.CapabilitiesPattern
0x0f4fa314     19,580       391,600 System.Web.Configuration.CapabilitiesAssignment
0x0f4f578c      4,932       572,112 System.Web.UI.WebControls.DataGridItem
0x00e331e8     24,624       728,648 System.Int32[]
0x79ba8cc8    119,338     1,432,056 System.Int32
0x00e334a8      6,058     1,984,248 System.Collections.Hashtable/bucket[]
0x0c0c8a7c    187,381     2,248,572 System.ComponentModel.EventHandlerList
0x0f4f864c    117,158     2,343,160 System.Web.UI.Triplet
0x0d814bb0    193,637     3,098,192 System.Web.UI.StateBag
0x0d963fbc    206,088     3,297,408 System.Web.UI.StateItem
0x10181dfc    228,298     3,652,768 System.Web.UI.Pair
0x0c0cb334    187,588     3,751,760 System.ComponentModel.EventHandlerList/ListEntry
0x0c14c620    193,747     3,874,940 System.Collections.Specialized.HybridDictionary
0x0c0c2cdc    206,354     4,127,080 System.Collections.Specialized.ListDictionary/DictionaryNode
0x0c0c2be8    193,728     4,649,472 System.Collections.Specialized.ListDictionary
0x79bc3564    188,036     5,265,008 System.EventHandler
0x79ba0d74    238,016     5,712,384 System.Collections.ArrayList
0x00e3292c        600     6,570,828 System.Byte[]
0x00e3209c    276,701    14,448,052 System.Object[]
0x00e3236c        490    17,826,392 System.Char[]
0x0f4f644c    187,416    17,991,936 System.Web.UI.WebControls.TableCell
0x105239f0    948,948    22,774,752 Test.TestDate
0x10559c20    243,243    35,026,992 Test.TestCollection
0x79b925c8  1,188,863   132,204,224 System.String
0x001516a0        122   226,349,940      Free
Total 5,288,099 objects, Total size: 523,832,112

 

From this, we can see that most of the memory is being taken up by System.String objects.  So there are a few ways we can look at the strings.  One of my favorite ways is to look at them grouped by size.  There is a switch to !dumpheap that that does that.  Here is what it looks like in this case:

 

 0:000> !clr10\sos.dumpheap -mt 0x79b925c8 -sizestat
Using our cache to search the heap.
   Address         MT     Size  Gen
Statistics:
        MT      Count     TotalSize Class Name
0x00009c40          3        34,044 
0x00002710         10        78,292 
0x00001388        230       332,184 
0x000001f4     78,748    15,744,792 
0x000186a0         83    23,806,908 
0x000003e8     80,049    42,256,244 
0x00000064  1,029,740    49,951,748 
Total 1,188,863 objects, Total size: 132,204,212

MT's are the upper bound of size for objects.
 0x00000064 = 100
    0x000001f4 = 500
    0x000003e8 = 1,000
  0x00001388 = 5,000
  0x00002710 = 10,000
 0x00009c40 = 40,000
 0x00014c08 = 85,000
 0x000186a0 = All above 85,000

 

This output is a little different, so the column listed at MT is actually the size of the object and then the count and total size in that range.  So now we can see that most of our strings are small and there are just a lot of them that are causing all the memory usage.  The next steps would be to look at the strings themselves, using !dumpheap -mt 0x79b925c8 (maybe use the -min and -max switches to limit it by size) and look for anything that jumps out.  The most common problem would be string concatenation.  This is usually seen with html strings where you would see one string that is "<table><tr>" and then the next string is "<table><tr><td>Data</td>" and so on, where each string builds on the next one.  This problem is easily fixed by using the System.StringBuilder class.  Take a look at the following articles for more information:

How to improve string concatenation performance in Visual C#
How to improve string concatenation performance in Visual Basic .NET or in Visual Basic 2005

Some of you may have noticed the large amount of memory that is being taken up by the Free object.  We will address that in a future posting.