Users can’t Login to ASP.net Application and get error “ Could not load file or assembly ‘—‘ or one of its dependencies. Access is denied.”

 This problem occurs intermittently and get resolved by Recycling Application pool and also it is very intermittent in which Assembly loose permissions.

 While the application was working very well intermittently users can’t login because ASP.net have lost permissions on one of the assemblies however it is loaded. Most of the time it had nothing to do with disk permissions it is more close to cross-AppDomain permissions, however the assembly is already loaded.

 Customer have collected a memory dumps at issue time however it is not the best option in this scenario:

 Dumps Details

 All users were trying to access Login.aspx and most of the time reply is 500

0:000> !aspxpages

Going to dump the HttpContexts found in the heap.

HttpContext Timeout Completed Running ThreadId ReturnCode Verb RequestPath+QueryString

0x01f0c224 300 Sec yes XXX 500 POST /DCR/login.aspx

….

0:000> !threads -special

ThreadCount: 7

UnstartedThread: 0

BackgroundThread: 7

PendingThread: 0

DeadThread: 0

Hosted Runtime: no

                                      PreEmptive GC Alloc Lock

  ID OSID ThreadOBJ State GC Context Domain Count APT Exception

  10 1 1c64 000d8b80 1808220 Enabled 0620124c:062025c8 000d2bd0 0 MTA (Threadpool Worker)

  14 2 3b84 000dd9f0 b220 Enabled 00000000:00000000 000d2bd0 0 MTA (Finalizer)

  15 3 3110 000f61f0 80a220 Enabled 00000000:00000000 000d2bd0 0 MTA (Threadpool Completion Port)

  16 4 3ca4 000f9648 1220 Enabled 00000000:00000000 000d2bd0 0 Ukn

   8 5 478c 00117410 880a220 Enabled 00000000:00000000 000d2bd0 0 MTA (Threadpool Completion Port)

  18 7 4628 00172370 180b220 Enabled 023da738:023dae00 000d2bd0 0 MTA (Threadpool Worker)

  23 6 3440 0fe5f8e0 880b220 Enabled 00000000:00000000 000d2bd0 0 MTA (Threadpool Completion Port)

       OSID Special thread type

    8 478c IOCompletion

    9 3968 Gate

   10 1c64 ThreadpoolWorker

   11 3df0 DbgHelper

   12 4138 GC

   13 2a24 GC

   14 3b84 Finalizer

   15 3110 Timer

   16 3ca4 ADUnloadHelper

   18 4628 ThreadpoolWorker

   23 3440 IOCompletion

 

GC and Finalizer Threads were In a  waiting state

 GC Threads

ChildEBP RetAddr 

0e08fd50 7c827b79 ntdll!KiFastSystemCallRet

0e08fd54 77e61d1e ntdll!NtWaitForSingleObject+0xc

0e08fdc4 79e8c5f9 kernel32!WaitForSingleObjectEx+0xac

0e08fe08 79e8c52f mscorwks!CLREventWaitHelper+0x2f

0e08fe58 79e8c54e mscorwks!CLREvent::WaitEx+0x117

0e08fe6c 79fcf272 mscorwks!CLREvent::Wait+0x17

0e08fe98 79f23b6a mscorwks!SVR::gc_heap::gc_thread_function+0x59

0e08ffb8 77e6482f mscorwks!SVR::gc_heap::gc_thread_stub+0x92

0e08ffec 00000000 kernel32!BaseThreadStart+0x34

 

Finalizer Thread

ChildEBP RetAddr 

0e0cfcc4 7c827b69 ntdll!KiFastSystemCallRet

0e0cfcc8 77e6202c ntdll!ZwWaitForMultipleObjects+0xc

0e0cfd70 77e62fbe kernel32!WaitForMultipleObjectsEx+0x11a

0e0cfd8c 79fd4762 kernel32!WaitForMultipleObjects+0x18

0e0cfdac 79fd4854 mscorwks!SVR::WaitForFinalizerEvent+0x77

0e0cfdc0 79e9848f mscorwks!SVR::GCHeap::FinalizerThreadWorker+0x49

0e0cfdd4 79e9842b mscorwks!ManagedThreadBase_DispatchInner+0x4f

0e0cfe68 79e98351 mscorwks!ManagedThreadBase_DispatchMiddle+0xb1

0e0cfea4 79f074d4 mscorwks!ManagedThreadBase_DispatchOuter+0x6d

0e0cfecc 79f074e5 mscorwks!ManagedThreadBase_NoADTransition+0x32

0e0cfedc 79f090b3 mscorwks!ManagedThreadBase::FinalizerBase+0xd

0e0cff14 79f75715 mscorwks!SVR::GCHeap::FinalizerThreadStart+0xbb

0e0cffb8 77e6482f mscorwks!Thread::intermediateThreadProc+0x49

0e0cffec 00000000 kernel32!BaseThreadStart+0x34

  !dumpheap –stat

………………………

0x79318184 1,665 33,300 Microsoft.Win32.SafeHandles.SafeTokenHandle

0x7932e9a8 766 36,768 System.Signature

0x6614adc8 192 36,864 System.Web.HttpContext

0x6614c868 192 37,632 System.Web.HttpResponse

0x7a5ed708 1,605 38,520 System.Collections.Specialized.ListDictionary+NodeEnumerator

0x66103798 2,490 39,840 System.Web.UI.StateItem

0x661018e8 993 43,692 System.Web.UI.Control+OccasionalFields

0x7a5eeb40 2,359 47,180 System.Collections.Specialized.HybridDictionary

0x6614e760 192 47,616 System.Web.Hosting.ISAPIWorkerRequestInProcForIIS6

0x79330c00 2,757 55,140 System.Text.StringBuilder

0x79332d10 4,729 56,748 System.Int32

0x66108aec 175 58,100 System.Web.UI.HtmlTextWriter+RenderAttribute[]

0x66101b9c 1,001 60,060 System.Web.UI.LiteralControl

0x0e6d5520 182 72,072 ASP.login_aspx

0x7a5eebfc 2,584 72,352 System.Collections.Specialized.ListDictionary

0x7932dcfc 1,604 89,824 System.Reflection.RuntimePropertyInfo

0x7a5eecbc 4,841 96,820 System.Collections.Specialized.ListDictionary+DictionaryNode

0x66149620 65 100,620 System.Web.Caching.ExpiresEntry[]

0x7932fd80 3,272 183,232 System.Reflection.RuntimeMethodInfo

0x79332af4 7,814 187,536 System.Collections.ArrayList

0x79333118 3,433 192,248 System.Collections.Hashtable

0x79332c60 1,873 213,424 System.Int32[]

0x793316f4 2,861 306,144 System.Char[]

0x79333214 3,446 606,696 System.Collections.Hashtable+bucket[]

0x79304294 18,943 1,140,248 System.Object[]

0x000da858 48 1,194,736 Free

0x79333534 3,619 1,596,536 System.Byte[]

0x79330ac4 30,958 3,527,892 System.String

Total 165,424 objects, Total size: 12,080,320

 

Dumping more in the ASP Object

!dumpheap –mt 0x0e6d5520       

             <b> Exception Details: </b>System.IO.FileLoadException: Could not load file or assembly Assembly NAME , Version=1.0.0.0, Culture=neutral, PublicKeyToken= -- or one of its dependencies. Access is denied.<br><br>

            …..

WRN: Assembly binding logging is turned OFF.

To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.

Note: There is some performance penalty associated with assembly bind failure logging.

To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

</pre></code>

 This module was already loaded  in the dump

0:014> lmvm ModuleNAME

start end module name

 ... Module Details …

 This is resolved by:

1. Confirm that the the account running Application pool have full permissions on ASP.net temporary folder (c:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files).

2. Remove the assembly “Assembly NAME .DLL” from the BIN folder and install it to the Global Assembly Cache. Then stop IIS, clear out “temporary asp.net files” and start IIS again. 

3. Disable\uninstall Antivirus if there is in the Webserver.

4. Enable fusion log (set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.)