Memory Leak in ASP.Net Application

 I started to be very interested in .net debugging and here is the best easiest source to understand the basics, where to start and also have some repo samples and labs J .NET Debugging Demos - Information and setup instructions

By Following some labs there you will know how to configure debugger and attach symbols and some psscor commands to know how to debug.  

I have come across interesting scenario of memory leak.

 The customer have collected 3 dumps for w3wp  at size 1.6 GB, I  have checked dumps and there are 3 main Problems that for behind memory size 1.6 GB:

  #1 Some dlls in debug Mode

0:000> !FindDebugModules

Loading all modules. Searching for modules built in debug mode... LONG LIST OF Modules.DLL not built release .… 

#2 Thousands of Data Set Tables are stored in memory and don’t get cleaned up and that cause Memory Fragmentation

The issue seems to be with the managed heap since there is a parallel increase in the managed heap objects:

!dumpheap -stat

0x725c234c 4,542 5,139,800 System.Int64[] 0x018bc4a0 90,779 6,536,088 AirportDS+airportRow 0x714ec760 361,866 7,237,320 System.Xml.XmlAttribute 0x714ea278 437,130 8,742,600 System.Xml.XmlText 0x725c35e0 2,013 8,922,616 System.Byte[] 0x725c2d0c 117,151 10,249,692 System.Int32[] 0x6fbac664 806,134 12,898,144 System.Web.Caching.CacheDependency+DepFileInfo 0x6e57055c 11,736 17,857,056 System.Data.RBTree`1+Node[[System.Int32, mscorlib]][] 0x6e56f4a0 16,355 20,348,580 System.Data.RBTree`1+Node[[System.Data.DataRow, System.Data]][] 0x6e56e6cc 142,636 21,110,128 System.Data.DataColumn 0x6fbab86c 806,188 22,573,264 System.Web.FileMonitorTarget 0x6fbab70c 800,627 25,620,064 System.Web.FileChangeEventHandler 0x6fbac5f4 800,740 28,826,640 System.Web.Caching.CacheDependency 0x714eb0b0 1,316 37,419,204 MS.Internal.Xml.Cache.XPathNode[] 0x01fb7410 2,223 37,667,012 Free 0x725c32c0 38,231 57,362,952 System.Collections.Hashtable+bucket[] 0x72594340 287,093 71,507,572 System.Object[] 0x725c0b70 6,082,917 973,634,772 System.String Total 13,119,720 objects, Total size: 1,463,157,524 Fragmented blocks larger than 0.5 MB: Addr Size Followed by ac9e3430 5.0MB acee38f0 System.Byte[] b210f048 2.5MB b2389c50 System.Byte[]

To Check All Data Tables

!DumpDataTables

 DataTable Rows Columns DataSet nextRowID ColumnCount 


0x7a8d5d8 0x7a8d890 0x7a8d7d0 0x89dfeb8 883 4 

0x6dbb130 0x6dbb3e8 0x6dbb328 0x89dfeb8 883 4 

0x6cbd180 0x6cbd438 0x6cbd378 0x89dfeb8 883 4 

0x6c74184 0x6c7443c 0x6c7437c 0x89dfeb8 883 4 

0x6bd6b8c 0x6bd6e44 0x6bd6d84 0x89dfeb8 883 4 

0x5852090 0x5852348 0x5852288 0x89dfeb8 883 4 

0x4c57708 0x4c579c0 0x4c57900 0x89dfeb8 883 4 

0x3bb062c 0x3bb08e4 0x3bb0824 0x89dfeb8 883 4 

0x39134d8 0x3913790 0x39136d0 0x89dfeb8 883 4 

0x369232c 0x36925e4 0x3692524 0x89dfeb8 883 4 

0x2c8eae0 0x2c8ed98 0x2c8ecd8 0x89dfeb8 883 4 

0x2bc3db8 0x2bc4070 0x2bc3fb0 0x89dfeb8 883 4 

0xf91f9d4 0xf91fce4 0xf91fb94 0xf91f88c 14,413 3 

Total 13,947 DataTable objects 

 

We can dig more and get columns names

0:000> !dcn 0x2bc3db8

 Column Names for DataTable: 0x2bc3db8 
 
…….. 

The number of data tables is huge! 14,413  Data Set is very big number to be stored in the memory and they don’t get cleaned up . Most of them have similar data, so we may apply caching mechanism and reuse the repeated values.  And dispose the Data objects after using it.

The size of free space has been increased in GEN2 heap, the issue seems to be with GEN2 heap:

!dumpheap -stat -gen 2

 0x01fb7410 619 6,412,056 Free 
0x714ec760 361,841 7,236,820 System.Xml.XmlAttribute 
0x714ea278 421,261 8,425,220 System.Xml.XmlText 

0x725c2d0c 115,433 10,076,824 System.Int32[] 

0x6fbac664 805,982 12,895,712 System.Web.Caching.CacheDependency+DepFileInfo 

0x725c32c0 29,377 16,599,312 System.Collections.Hashtable+bucket[] 

0x6e57055c 11,712 17,810,688 System.Data.RBTree`1+Node[[System.Int32, mscorlib]][] 

0x6e56f4a0 16,280 20,249,376 System.Data.RBTree`1+Node[[System.Data.DataRow, System.Data]][] 

0x6e56e6cc 141,873 20,997,204 System.Data.DataColumn 

0x6fbab86c 806,036 22,569,008 System.Web.FileMonitorTarget 

0x714eb0b0 1,126 23,513,992 MS.Internal.Xml.Cache.XPathNode[] 

0x6fbab70c 800,475 25,615,200 System.Web.FileChangeEventHandler 

0x6fbac5f4 800,588 28,821,168 System.Web.Caching.CacheDependency 

0x72594340 272,768 70,031,728 System.Object[] 

0x725c0b70 5,836,643 511,657,644 System.String 

Total 12,522,704 objects, Total size: 897,063,028 


 

Conclusion, these data tables in GEN2 heap are not cleaned up and promoted to GEN2, after a cleanup, memory is fragmented in

GEN2 heap. But there are possibly new other objects (datatables) which are promoted from GEN 1 to GEN2 , that is causing the memory to grow.

You may have a look at that same type of data tables, and may prefer to keep a single instance if the data inside these tables are the same

  #3 Number of Objects System.Web.FileChangeEventHandler and System.Web.Caching.CacheDependency are growing till reach million in last dump

From the other side there is a 200,000 increase on number of System.Web.FileChangeEventHandler and System.Web.Caching.CacheDependency objects , that may be an evidence of increasing GEN2 usage

 

(System.Web.HttpRuntime)-> 
0c2991b0(System.Web.FileChangesMonitor)-> 

0c299298(System.Collections.Hashtable)-> 

0407c87c(System.Collections.Hashtable+bucket[])-> 
0ac87f00(System.Web.DirectoryMonitor)-> 

0ac87f24(System.Collections.Hashtable)-> 

0ac87f5c(System.Collections.Hashtable+bucket[])-> 

0ac88074(System.Web.FileMonitor)-> 

0ac880a8(System.Collections.Specialized.HybridDictionary)-> 

04e33fa8(System.Collections.Hashtable)-> 

a1fe0038(System.Collections.Hashtable+bucket[])-> 

028f2ed8(System.Web.Caching.CacheDependency) 




 

 So Caching Objects are for reuse not creating new ones over and over ..