Debugging an intermittent freeze issue in a winforms application, managed memory leak detection using SOS: Methodology and Approach


Hello, I am back to my old habits of blogging about some interesting things that we do. So recently we were approached by one of our important partners regarding a Freeze in their Winforms application which was ‘intermittent’. This freeze or hang per say was accompanied with the below exception, 

Exception: System.ComponentModel.Win32Exception
Message: Error creating window handle.
 

It was pretty evident from the error message that we are unable to create handles and one other aspect of the issue was that we really had high no of objects in the Task manager. An important point to note here is that we do not see a crash in the
application. To make sure we have the right data, I had to capture a first chance exception dump on Win32Exception. For more information on First chance exception please read the Blog. There are loads of tools you could use to capture this dump, DebugDiag is an user friendly UI tool which helps you fiddle around with exception names. Considering the issue to be intermittent Debugdiag is an ideal tool in this scenario as all we have to do is to set up a rule and DebugDiag will take  care of the rest. While the DebugDiag UI is pretty straight forward , Once you open DebugDiag , Click on Add Rule->Crash->Click Next-> A Specific Process(This feature to be used for exes)->Now Select the process (from the process list or write the Application name if it is not running)-> Click Next . On this windows I would add specific actions considering I want a first chance dump for a particular exception. Now click on Exceptions->Add Exception. Further Select E0434352 and add the corresponding details as shown below.

 

After all the above steps , it is important to RESTART YOUR APPLICATION. 

Well after the above steps we did receive the dump file and here is how we nailed down the issue. Excerpts from analysis below. I believe windbg is a powerful debugger and I prefer to use it, any other debugger can be used to debug the dumps. Visual Studio has many inbuilt feature for memory analysis , a very handy tool to be used in High memory scenarios.

0:000> !pe
Exception object: 024c96bc
Exception type: System.ComponentModel.Win32Exception
Message: Error creating window handle.
InnerException: <none>
StackTrace (generated):
SP IP Function
001DED08 6692CF75 System_Windows_Forms_ni!System.Windows.Forms.NativeWindow.CreateHandle(System.Windows.Forms.CreateParams)+0x2c5
001DED80 6692CBA5 System_Windows_Forms_ni!System.Windows.Forms.Control.CreateHandle()+0x115
001DEDDC 6692E9FC System_Windows_Forms_ni!System.Windows.Forms.Control.get_Handle()+0x4c
001DEDEC 668DB724 System_Windows_Forms_ni!System.Windows.Forms.Control.CreateGraphicsInternal()+0x8
001DEDF4 67090D38 System_Windows_Forms_ni!System.Windows.Forms.ThreadExceptionDialog..ctor(System.Exception)+0x800
001DEEF8 66E8EA1E System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.OnThreadException(System.Exception)+0xb6
001DEF34 66E7A782 System_Windows_Forms_ni!System.Windows.Forms.Control.WndProcException(System.Exception)+0x16
001DEF40 66E7BA6A System_Windows_Forms_ni!System.Windows.Forms.Control+ControlNativeWindow.OnThreadException(System.Exception)+0xa
001DEF44 6692E18B System_Windows_Forms_ni!System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)+0x9b
00000000 00000001 System_Windows_Forms_ni!System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG ByRef)+0x2
001DF1E0 6693DCA1 System_Windows_Forms_ni!System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr, Int32, Int32)+0x24d
001DF26C 6693D921 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext)+0x155
001DF2BC 6693D792 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)+0x4a
001DF2E8 668E14BD System_Windows_Forms_ni!System.Windows.Forms.Application.Run(System.Windows.Forms.Form)+0x31
001DF2FC 00414203 Example1!Example1.Program.Main(System.String[])+0x143

The above exception is triggered, we have to find our which user objects are being used that lead to this.
Although the nature of the objects occupying memory may vary , we have to understand that we should
only be worried about UI objects
precisely which would make the application throw such an error, namely
Dialogs, Windows, timers, Ui controls etc. We will use a command !address -summary to find out which
objects occupy what amount of memory.
MT Count Size Class Name
66974944 619 9904 System.Windows.Forms.PropertyStore+SizeWrapper
68097098 226 9944 System.Reflection.Emit.OpCode
66975558 242 10344 System.Windows.Forms.PropertyStore+IntegerEntry[]
680ccfc8 6 10952 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[System.String, mscorlib]][]
6ab68ea0 463 20372 System.Drawing.Font
004298e4 588 21168 QSYS.MSG.MessageText
680d8b40 898 21552 System.Collections.ArrayList
67589644 229 28396 System.ComponentModel.ReflectPropertyDescriptor
6808ef70 521 29176 System.Reflection.RuntimePropertyInfo
6758749c 1525 30500 System.ComponentModel.EventHandlerList+ListEntry
669749f8 591 30732 System.Windows.Forms.CreateParams
680dd288 598 31096 System.Collections.Hashtable
680db970 82 32140 System.Char[]
6a975f6c 584 32704 System.Configuration.ConfigurationProperty
669743e0 592 33152 System.Windows.Forms.Control+ControlNativeWindow
08e31cc4 355 39760 com.ibsag.qsys.svr.system.XDQSFormItem
680dc738 335 42496 System.Int32[]
6697497c 602 51124 System.Windows.Forms.PropertyStore+ObjectEntry[]
08e32b80 5 56100 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[QSYS.WindowsControls.BL.LayoutTranslationEngine+LayoutItem, QSYS_WindowsControls]][]
680cee64 838 56984 System.Reflection.RuntimeParameterInfo
680d7618 1338 58872 System.Signature
00b02918 348 64032 QSYS.WindowsControls.IBSControls.Button
680d8ce0 5899 70788 System.WeakReference
680db350 6080 72960 System.Object
680dae80 599 76224 System.Collections.Hashtable+bucket[]
680d7940 1529 91740 System.Reflection.RuntimeMethodInfo
6697476c 1 101040 System.Windows.Forms.NativeWindow+HandleBucket[]
680dc168 5157 144396 System.RuntimeType
66975130 5110 163520 System.Windows.Forms.NativeMethods+WndProc
680d08a0 5535 177120 System.EventHandler
66961308 4976 238848 System.Windows.Forms.Timer
6696306c 4970 258440 System.Windows.Forms.Timer+TimerNativeWindow
6808ae88 5854 369200 System.Object[]
004bbc98 15 565718 Free
680dafb0 14211 911970 System.String
680dd244 501 958102 System.Byte[]
6808e948 94719 3031008 System.Globalization.DaylightTime
Total 192318 objects
Based on the above summary it is very clear that for now we will have to ignore System.String objects and 
concentrate on
these UI objects. Having known the highlighted objects occupy a fair amount of memory
we can look into our code to see
where we use these objects and see if we are leaking memory somewhere.
We can further debug in the memory dumps to
find out the exact roots of these objects and track them
accordingly, So I set out to find out the roots for alacrity to our
customer as to where exactly does he have
to look into. I would dump the objects using !dumpheap -mt Methodtable to
dump the below info.
02423764 66961308  48 
02423a54 66961308 48
.
.
02425dac 66961308 48
02426144 66961308 48 
024261e0 66961308 48
0242627c 66961308 48
.
.
02426f44 66961308 48
02426fe0 66961308 48
0242707c 66961308 48
02427118 66961308 48
02428938 66961308 48
.
.
0242baa0 66961308 48 
0242bb3c 66961308 48
0242bbf8 66961308 48
0242bc94 66961308 48
0242bd50 66961308 48
0242c118 66961308 48
0242c224 66961308 48
0242c2c0 66961308 48
0242c35c 66961308 48
0242c3f8 66961308 48
0242c4b4 66961308 48
0242c550 66961308 48
0242c764 66961308 48
0242c82c 66961308 48
0242c9c4 66961308 48
0242cab4 66961308 48
02434f28 66961308 48
02435010 66961308 48

Statistics:
MT Count TotalSize Class Name
66961308 4976 238848 System.Windows.Forms.Timer

0:000> !gcroot -nostacks 02435010
HandleTable:
001f1130 (strong handle)
-> 02353720 System.Windows.Forms.Timer
-> 0235375c System.EventHandler
-> 0205d524 SPCInsp.MainFormSPCInsp
-> 0217de18 SPCInsp.LoginCenter
-> 02434fe4 SPCInsp.AutoPnrHandler
-> 02435010 System.Windows.Forms.Timer
You can observe that we have found the roots at 02435010. I can confirm that most of the other roots were very 
synonymous.
Further we should look at the other type of objects from what we had yellow marked from
!address -Summary, let’s look
at the other object in a similar way.
 
024388b4 66975130 32 
.
.
02448874 66975130 32
02448a3c 66975130 32
02448a5c 66975130 32
02448a7c 66975130 32
02448ad8 66975130 32
02448b34 66975130 32
02448b90 66975130 32
02448be4 66975130 32
02448c40 66975130 32
02448c60 66975130 32
.
.
024c6500 66975130 32

024c6520 66975130 32
024c6540 66975130 32
024c6560 66975130 32
024c6580 66975130 32
024c65dc 66975130 32
024c6610 66975130 32
024c6630 66975130 32
024c6650 66975130 32
024c66ac 66975130 32
024c66cc 66975130 32
024c66ec 66975130 32
024c670c 66975130 32
024c672c 66975130 32
024c674c 66975130 32
024c67a8 66975130 32
024c70f0 66975130 32
024c7110 66975130 32
024c7130 66975130 32
024c7150 66975130 32
024c7170 66975130 32
024c7288 66975130 32
024c72a8 66975130 32
024c72c8 66975130 32
024c72e8 66975130 32
024c7308 66975130 32
024c7328 66975130 32
024c7348 66975130 32
024c7368 66975130 32

Randomly dumping the objects with roots,

0:000> !gcroot -nostacks 024c7368
HandleTable:
001f1130 (strong handle)
-> 02353720 System.Windows.Forms.Timer
-> 0235375c System.EventHandler
-> 0205d524 Example1.MainFormExample1
-> 02435378 Example2.WindowsControls.Controls.Capturing.Inspections.Inspections
-> 024356dc System.Collections.ArrayList
-> 0248c8ec System.Object[]
-> 02446224 Example2.WindowsControls.Example3.ComboBox
-> 024463d4 System.ComponentModel.EventHandlerList
-> 02487ab8 System.ComponentModel.EventHandlerList+ListEntry
-> 02487aa4 System.ComponentModel.EventHandlerList+ListEntry
-> 02487a90 System.ComponentModel.EventHandlerList+ListEntry
-> 02487a7c System.ComponentModel.EventHandlerList+ListEntry
-> 02487a68 System.ComponentModel.EventHandlerList+ListEntry
-> 02487a54 System.ComponentModel.EventHandlerList+ListEntry
-> 024484d4 System.ComponentModel.EventHandlerList+ListEntry
-> 02448124 System.ComponentModel.EventHandlerList+ListEntry
-> 0248abbc System.EventHandler
-> 0248ab8c System.Object[]
-> 02448104 System.EventHandler
-> 02444bf4 Example2.WindowsControls.Controls.Capturing.AcquisitionControl.SearchPage
-> 02445008 System.ComponentModel.EventHandlerList
-> 02487a40 System.ComponentModel.EventHandlerList+ListEntry
-> 02487a2c System.ComponentModel.EventHandlerList+ListEntry
-> 02487a18 System.ComponentModel.EventHandlerList+ListEntry
-> 02487a04 System.ComponentModel.EventHandlerList+ListEntry
-> 024879f0 System.ComponentModel.EventHandlerList+ListEntry
-> 024491d8 System.ComponentModel.EventHandlerList+ListEntry
-> 024cb554 System.Windows.Forms.MouseEventHandler
-> 024cb534 System.Object[]
-> 024491b8 System.Windows.Forms.MouseEventHandler
-> 0244463c Example2.WindowsControls.Controls.Capturing.AcquisitionControl.Acquisition
-> 024448ec System.Windows.Forms.Control+ControlNativeWindow
-> 024c7368 System.Windows.Forms.NativeMethods+WndProc

Found 1 unique roots (run '!GCRoot -all' to see all roots).
0:000> !gcroot -nostacks 024c5de0
HandleTable:
001f1130 (strong handle)
-> 02353720 System.Windows.Forms.Timer
-> 0235375c System.EventHandler
-> 0205d524 Example1.MainFormExample1
-> 02435378 Example2.WindowsControls.Controls.Capturing.Inspections.Inspections
-> 024c5c28 System.Windows.Forms.LayoutEventArgs
-> 02482048 Infragistics.Win.UltraWinTabControl.UltraTabControl
-> 02489f2c Infragistics.Win.UltraWinTabControl.UltraTabsCollection
-> 02489fac System.Collections.ArrayList
-> 02489fc4 System.Object[]
-> 0243570c Infragistics.Win.UltraWinTabControl.UltraTab
-> 024358e0 Infragistics.Win.UltraWinTabControl.UltraTabPageControl
-> 024c5da4 Infragistics.Win.UIAutomation.UiaProviderControlNativeWindow
-> 024c5de0 System.Windows.Forms.NativeMethods+WndProc

Found 1 unique roots (run '!GCRoot -all' to see all roots).
0:000> !gcroot -nostacks 02483e7c
HandleTable:
001f1130 (strong handle)
-> 02353720 System.Windows.Forms.Timer
-> 0235375c System.EventHandler
-> 0205d524 Example1.MainFormExample1
-> 02435378 Example2.WindowsControls.Controls.Capturing.Inspections.Inspections
-> 024356dc System.Collections.ArrayList
-> 0248c8ec System.Object[]
-> 02477260 Example2.WindowsControls.Example3.ComboBox
-> 02477410 System.ComponentModel.EventHandlerList
-> 02489638 System.ComponentModel.EventHandlerList+ListEntry
-> 02489624 System.ComponentModel.EventHandlerList+ListEntry
-> 02489610 System.ComponentModel.EventHandlerList+ListEntry
-> 024895fc System.ComponentModel.EventHandlerList+ListEntry
-> 024895e8 System.ComponentModel.EventHandlerList+ListEntry
-> 024895d4 System.ComponentModel.EventHandlerList+ListEntry
-> 02479510 System.ComponentModel.EventHandlerList+ListEntry
-> 02479160 System.ComponentModel.EventHandlerList+ListEntry
-> 0248c2bc System.EventHandler
-> 0248c28c System.Object[]
-> 02479140 System.EventHandler
-> 02475c50 Example2.WindowsControls.Controls.Capturing.AcquisitionControl.SearchPage
-> 02476064 System.ComponentModel.EventHandlerList
-> 024895c0 System.ComponentModel.EventHandlerList+ListEntry
-> 024895ac System.ComponentModel.EventHandlerList+ListEntry
-> 02489598 System.ComponentModel.EventHandlerList+ListEntry
-> 02489584 System.ComponentModel.EventHandlerList+ListEntry
-> 02489570 System.ComponentModel.EventHandlerList+ListEntry
-> 0247a178 System.ComponentModel.EventHandlerList+ListEntry
-> 024cc60c System.Windows.Forms.MouseEventHandler
-> 024cc5ec System.Object[]
-> 0247a158 System.Windows.Forms.MouseEventHandler
-> 02475698 Example2.WindowsControls.Controls.Capturing.AcquisitionControl.Acquisition
-> 02475a4c System.ComponentModel.EventHandlerList
-> 024894e4 System.ComponentModel.EventHandlerList+ListEntry
-> 024894d0 System.ComponentModel.EventHandlerList+ListEntry
-> 024894bc System.ComponentModel.EventHandlerList+ListEntry
-> 024894a8 System.ComponentModel.EventHandlerList+ListEntry
-> 02489494 System.ComponentModel.EventHandlerList+ListEntry
-> 02481638 System.ComponentModel.EventHandlerList+ListEntry
-> 024cc58c System.Windows.Forms.MouseEventHandler
-> 024cc574 System.Object[]
-> 02481618 System.Windows.Forms.MouseEventHandler
-> 02471e64 Example2.WindowsControls.Controls.Capturing.MontageCapturing.MontageCapturing
-> 024721b4 System.Collections.ArrayList
-> 02481848 System.Object[]
-> 0247af90 Example2.WindowsControls.Example3.InfraTextEditor
-> 0247b924 System.ComponentModel.EventHandlerList
-> 02489b9c System.ComponentModel.EventHandlerList+ListEntry
-> 02489b88 System.ComponentModel.EventHandlerList+ListEntry
-> 02489b74 System.ComponentModel.EventHandlerList+ListEntry
-> 02483d6c System.ComponentModel.EventHandlerList+ListEntry
-> 0247fbb0 System.ComponentModel.EventHandlerList+ListEntry
-> 0247fb7c System.ComponentModel.EventHandlerList+ListEntry
-> 0247fb48 System.ComponentModel.EventHandlerList+ListEntry
-> 0247f480 System.ComponentModel.EventHandlerList+ListEntry
-> 0247f40c System.ComponentModel.EventHandlerList+ListEntry
-> 0247bb60 System.ComponentModel.EventHandlerList+ListEntry
-> 0247b95c System.ComponentModel.EventHandlerList+ListEntry
-> 0247b948 System.ComponentModel.EventHandlerList+ListEntry
-> 0247f440 System.Windows.Forms.ControlEventHandler
-> 0247a8bc Example2.WindowsControls.Controls.Capturing.MontageCapturing.MontageEntry
-> 02486000 System.Windows.Forms.LayoutEventArgs
-> 0247aca8 Example2.WindowsControls.Example3.GroupBox
-> 02485ff0 System.Windows.Forms.LayoutEventArgs
-> 0247adc8 Example2.WindowsControls.Example3.TableLayoutPanel
-> 0247ae84 System.Windows.Forms.PropertyStore
-> 02481170 System.Windows.Forms.PropertyStore+ObjectEntry[]
-> 0247ee28 System.Windows.Forms.Layout.TableLayout+ContainerInfo
-> 0247ff68 System.Object[]
-> 0247f044 System.Windows.Forms.Layout.TableLayout+LayoutInfo
-> 0247c1dc Example2.WindowsControls.Example3.InfraTextEditor
-> 02483e40 Infragistics.Win.UltraWinEditors.TextEditorControlBase+TextEditorControlBaseNativeWindow
-> 02483e7c System.Windows.Forms.NativeMethods+WndProc

Found 1 unique roots (run '!GCRoot -all' to see all roots).
0:000> !gcroot -nostacks 02479c60
HandleTable:
001f1130 (strong handle)
-> 02353720 System.Windows.Forms.Timer
-> 0235375c System.EventHandler
-> 0205d524 Example1.MainFormExample1
-> 02435378 Example2.WindowsControls.Controls.Capturing.Inspections.Inspections
-> 024356dc System.Collections.ArrayList
-> 0248c8ec System.Object[]
-> 02477260 Example2.WindowsControls.Example3.ComboBox
-> 02477410 System.ComponentModel.EventHandlerList
-> 02489638 System.ComponentModel.EventHandlerList+ListEntry
-> 02489624 System.ComponentModel.EventHandlerList+ListEntry
-> 02489610 System.ComponentModel.EventHandlerList+ListEntry
-> 024895fc System.ComponentModel.EventHandlerList+ListEntry
-> 024895e8 System.ComponentModel.EventHandlerList+ListEntry
-> 024895d4 System.ComponentModel.EventHandlerList+ListEntry
-> 02479510 System.ComponentModel.EventHandlerList+ListEntry
-> 02479160 System.ComponentModel.EventHandlerList+ListEntry
-> 0248c2bc System.EventHandler
-> 0248c28c System.Object[]
-> 02479140 System.EventHandler
-> 02475c50 Example2.WindowsControls.Controls.Capturing.AcquisitionControl.SearchPage
-> 02476abc Infragistics.Win.UltraWinEditors.UltraOptionSet
-> 02476bd4 System.Windows.Forms.Control+ControlNativeWindow
-> 02479c60 System.Windows.Forms.NativeMethods+WndProc

Found 1 unique roots (run '!GCRoot -all' to see all roots).
0:000> !gcroot 0242d0d0
HandleTable:
001f1130 (strong handle)
-> 02353720 System.Windows.Forms.Timer
-> 0235375c System.EventHandler
-> 0205d524 Example1.MainFormExample1
-> 0242cbd0 Infragistics.Win.UltraWinTabControl.UltraTab
-> 0242ce2c Infragistics.Win.UltraWinTabControl.UltraTabPageControl
-> 0242d094 Infragistics.Win.UIAutomation.UiaProviderControlNativeWindow
-> 0242d0d0 System.Windows.Forms.NativeMethods+WndProc

Found 1 unique roots (run '!GCRoot -all' to see all roots).

With all the above objects our customer had handful objects to look at , what we eventually found out is that the timer object was created multiple times, even if it already existed . The timer was not stopped, and the tick event not unsubscribed, so here was the problem. They eventually made sure that timer object was created only once. Post this change we observed that there was no problem with the memory and there was no freeze.

Keep Coding!
Nandeesh Swami

 


Comments (0)

Skip to main content