W3WP.EXE hangs with 50% CPU usage


 

This is an ASP.Net application running on IIS 7. The application hangs with 50% CPU every time making a change to web.config. Customer has to restart IIS services to resolve the problem.

Changing web.config during application running is not a good idea since this triggers application domain recycle. This used to cause short time performance slow down; however, the application should not hang forever. Again, we get 2 hang dumps with 3 minutes interval to investigate this issue.

First of all, we compared the "runaway" result of these 2 dumps, definitely, thread 99c is the culprit.

0:019> !runaway

User Mode Time

  Thread       Time

  19:99c       0 days 0:06:24.906

  17:df8       0 days 0:00:00.656

  ..........

 

0:019> !runaway

User Mode Time

  Thread       Time

  19:99c       0 days 0:09:46.125

  17:df8       0 days 0:00:00.656

  ............

 Thread 99c is the finalizer thread. It in both 2 dumps, it had similar call stacks.

0:019> kc

mscorwks!Thread::SafeSetLastThrownObject

mscorwks!Thread::EnterContextRestricted

mscorwks!Thread::DoADCallBack

mscorwks!ManagedThreadBase_DispatchInner

mscorwks!ManagedThreadBase::FinalizerAppDomain

mscorwks!SVR::DoOneFinalization

mscorwks!SVR::FinalizeAllObjects

mscorwks!SVR::GCHeap::FinalizerThreadWorker

mscorwks!ManagedThreadBase_DispatchInner

mscorwks!ManagedThreadBase_DispatchMiddle

mscorwks!ManagedThreadBase_DispatchOuter

mscorwks!ManagedThreadBase_NoADTransition

mscorwks!ManagedThreadBase::FinalizerBase

mscorwks!SVR::GCHeap::FinalizerThreadStart

........

 So, looks the finalizer thread has trouble with some objects. The object to be finalized is the 3rd parameter of "mscorwks!ManagedThreadBase::FinalizerAppDomain"

 0:019> dt 0x421f93c  WKS::FinalizeAllObjects_Args

clr!WKS::FinalizeAllObjects_Args

   +0x000 fobj             : 0x02273ec0 Object

   +0x004 bitToCheck       : 0n0

 

This looks a simple object.

0:019>!do 02273ec0

Name:        ChildClass

MethodTable: 01450690

EEClass:     011665f4

Size:        24(0x18) bytes

File:        C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\reregisterforfinalize\1a059407\bda2ac57\App_Code.wwiz6uz0.dll

Fields:

      MT    Field   Offset                 Type VT     Attr    Value Name

6f11f9ac  4000001        4        System.String  0 instance 02273e24 _name

6f11f9ac  4000002        8        System.String  0 instance 02273e5c _address

6f11f9ac  4000003        c        System.String  0 instance 02273e8c _postcode

6f11f9ac  4000004       10        System.String  0 instance 02273e3c _phone

 And the parent class implemented Finalize method.

0:019> !dumpmt -md 01450690

EEClass:      011665f4

Module:       01026274

Name:         ChildClass

mdToken:      02000003

File:         C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\reregisterforfinalize\1a059407\bda2ac57\App_Code.wwiz6uz0.dll

BaseSize:        0x18

ComponentSize:   0x0

Slots in VTable: 6

Number of IFaces in IFaceMap: 1

--------------------------------------

MethodDesc Table

   Entry MethodDesc      JIT Name

6f02a7e0   6ee04934   PreJIT System.Object.ToString()

6f02e2e0   6ee0493c   PreJIT System.Object.Equals(System.Object)

6f02e1f0   6ee0495c   PreJIT System.Object.GetHashCode()

014110d0   01450600      JIT BaseClass.Finalize()

0102c251   014505f8     NONE BaseClass.Dispose()

0102c270   0145067c     NONE ChildClass..ctor()

 

Look at the code of Finalize method; we could see it simply calls GC.ReRegisterForFinalize.

~BaseClass()

{

    GC.ReRegisterForFinalize(this);

}

 

So, what's wrong here?

The finalization method for an object provides you an opportunity to perform any necessary cleanup operations before the object is garbage collected. After finalization, the object is accessible but in an invalid state and therefore unusable. Eventually, garbage collection completes and reclaims the object.

An object's finalization method is called in one of the following situations:

  • - During garbage collection,
  • - When the application domain that contains the object is unloaded,
  • - When the common language runtime is shutting down.

For code above when the finalizer is executing if a new reference to the object is created and GC.ReRegisterForFinalize is called then the object becomes reachable and hence non garbage and so in the GC will not reclaim it. This sometime is useful if the creation of the object is expensive.

 Then, again, what's wrong with this code? During the unloading of application domain, all objects belongs to the domain will be cleared. This code simply add the object back to finalize queue and results in the finalize thread calls its finalize method again and again.

Follow code should be used to resolve this problem.

~BaseClass(){

if (!Environment.HasShutdownStarted)

{

 GC.ReRegisterForFinalize(this);

}

}

 

At last, why it shows 50% CPU instead of 100%? I think this was due to the application is running on a dual core CPU system. I am able to reproduce this issue on a dual core CPU machine; it shows 50% CPU usage. However, on one of my out of date single core CPU machine, it shows 100% CPUJ

NOTE: this problem can be reproduced on both .Net 2.0 and .Net 4.0.

 See you next time.

 Best Regards,

Wei

Comments (0)

Skip to main content