Debugging Hang issue of ASP.net 2.0 App running on IIS 6.


This Dump was very tricky and intersting, the issue was intermitently hange in ASP.net 2.0 Application running on IIS 6.

I checked dumps and found all threads are waiting on DB by two commands:

“~* e !clrstack” and “~ * e KL”

but they were all victim of thread 75

0:075> !syncblk
Index SyncBlock MonitorHeld Recursion Owning Thread Info  SyncBlock Owner
   50 0010467c           65         1 227cc860  19e4  75   02986720 System.Object
  Waiting threads: 34 36 39 40 41 44 45 46 47 48 52 53 54 55 56 60 62 66 67 69 71 72 78 80 83 85 86 91 92 93 94 96
—————————–
Total           256
CCW             2
RCW             0
ComClassFactory 0
Free            120

Here is thread 75

Managed Part
=============
0:075> !clrstack
OS Thread Id: 0x19e4 (75)
ESP       EIP    
258ce2a4 7c82845c [HelperMethodFrame: 258ce2a4]
258ce2fc 792e457e System.Collections.ArrayList.GetEnumerator()
258ce308 7a59deb5 System.Diagnostics.TraceInternal.WriteLine(System.String)
258ce344 7a5837c5 System.Diagnostics.Trace.WriteLine(System.String)
258ce348 1b6369ec ModuleNAME.Database.sqlGetFirst(System.String, System.Data.IDbConnection, System.Data.IDbTransaction)
258ce3f0 1b63675e ModuleNAME.Database.sqlGetFirst(System.String, System.Data.IDbConnection)
258ce408 1b636707 ModuleNAME.Database.sqlGetFirst(System.String)
258ce424 1b636591 PortalSite.Portal.Security.HasViewPermission(System.String)
258ce474 1b634c8e PortalSite.Portal.Engine.PageLoad(System.Web.UI.Page, Boolean)
258ce548 1b634bfc PortalSite.Portal.Engine.PageLoad(System.Web.UI.Page)
258ce554 1b634ba5 PortalSite.DefaultArthro.Page_Load(System.Object, System.EventArgs)
258ce564 66f2a7ff System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)
258ce574 660b4cc4 System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)
258ce588 660ae1c4 System.Web.UI.Control.OnLoad(System.EventArgs)
258ce59c 660ae203 System.Web.UI.Control.LoadRecursive()
258ce5b4 660aa2b4 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
258ce70c 660a9ee4 System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
258ce744 660a9e11 System.Web.UI.Page.ProcessRequest()
…………………………

Unmanaged Part
=============
0:075> KL
ChildEBP RetAddr 
258cdff0 7c827b79 ntdll!KiFastSystemCallRet
258cdff4 77e61d06 ntdll!NtWaitForSingleObject+0xc
258ce064 79e7552f kernel32!WaitForSingleObjectEx+0xac
258ce0a8 79e7547b mscorwks!CLREventWaitHelper+0x2f
258ce0f8 79e7549a mscorwks!CLREvent::WaitEx+0x117
258ce10c 79f0ee81 mscorwks!CLREvent::Wait+0x17
258ce130 79f1342e mscorwks!SVR::gc_heap::wait_for_gc_done+0x62
258ce1b8 79ead906 mscorwks!SVR::gc_heap::try_allocate_more_space+0x19
258ce1d4 79ead9fe mscorwks!SVR::gc_heap::allocate_more_space+0x2e
258ce1f4 79e72321 mscorwks!SVR::GCHeap::Alloc+0x54
258ce210 79e796e9 mscorwks!Alloc+0x60
258ce250 79e797ae mscorwks!FastAllocateObject+0x38
258ce2f4 792e457e mscorwks!JIT_NewFast+0x9e
258ce33c 7a59deb5 mscorlib_ni!System.Collections.ArrayList.GetEnumerator()+0xe
258ce33c 7a5837c5 System_ni!System.Diagnostics.TraceInternal.WriteLine(System.String)+0x75
258ce3e4 1b6369ec System_ni!System.Diagnostics.Trace.WriteLine(System.String)+0x5
258ce3e4 1b63675e ModuleNAME!.Database.sqlGetFirst(System.String, System.Data.IDbConnection, System.Data.IDbTransaction)+0x264
258ce400 1b636707 ModuleNAME!.Database.sqlGetFirst(System.String, System.Data.IDbConnection)+0x2e
258ce41c 1b636591 ModuleNAME.Database.sqlGetFirst(System.String)+0x37
……….

I wrote All Modules

 !sam c:\dlls\AllModules

And decombiled  ModuleNAME.Database.sqlGetFirst

Conclusion
==========
Application hang because all threads were waiting for the tracing to be written of methed sqlGetFirst. So we may fix code of ModuleNAME.Database.sqlGetFirst And  remove tracing module.

 

Comments (0)

Skip to main content