Attempt to release mutex not owned by caller Exception when Output Cache is enabled

After enabling Output Cache for site collections, you get the following error when browsing to it:

"Error in '/'
Application" message referring to an 'Attempt to release mutex not owned by caller.
(Exception from HRESULT: 0x00000120)'

Description: An unhandled exception occurred during the execution of the current
web request. Please review the stack trace for more information about the error end
where originated in the code.
Exception Details: SystemApplicationException: Attempt to release mutex not owned
by caller. (Exception from HRESULT: 0x00000120)
Source Error:
An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can be
identified using the exception stack trace below.
Stack Trace:
ApplicationException: Attempt to release mutex not owned by caller. (Exception from
HRESULT: 0x00000120))
System.Threading. ReaderwriterLock. ReleaseWriterLocklnternal () +0
Microsoft.SharePoint.Publishing.ThreadSafeCache '2.ReleasewriterLock() +73
Microsoft.SharePoint.Publishing.AclCache. LoadAllAcls(SPSite site) +715
Microsoft. SharePoint nt. Publishing.
PublishingHttpModule.EnableCachinglfAppropriate(Object source, EventArgs e) +4355
System.Web. SyncEventExecutionStep. System.Web. HttpApplication. lExecutionStep.
Execute() +80
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean&
completedSynchronously) +177

Also, you will notice the following event being thrown in the Event Viewer:

Event Type: Warning
Event Source: ASP.NET 2.0.50727.0
Event Category: Web Event
Event ID: 1309
Date: 1/7/2009
Time: 9:11:56 AM
User: N/A
Computer: <computer name>
Description:
Event code: 3005
Event message: An unhandled exception has occurred.
Event time: 1/7/2009 9:11:56 AM
Event time (UTC): 1/7/2009 2:11:56 PM
Event ID: 662477544e324e6da0b5b1007fd46f89
Event sequence: 2165
Event occurrence: 43
Event detail code: 0
Application information:
Application domain: /LM/W3SVC/2127404773/Root-1-128758103422019078
Trust level: wss_custom
Application Virtual Path: /
Application Path: C:\Inetpub\wwwroot\wss\VirtualDirectories\80\
Machine name: <computer name>
Process information:
Process ID: 6356
Process name: w3wp.exe
Account name: <domain\username>
Exception information:
Exception type: ApplicationException
Exception message: Attempt to release mutex not owned by caller. (Exception
from HRESULT: 0x00000120)
Request information:
Request URL: <full url>
Request path: <relative url>
User host address: <IP address>
User: <service account>
Is authenticated: True
Authentication Type: NTLM
Thread account name: <domain\username>
Thread information:
Thread ID: 7
Thread account name: <domain\username>
Is impersonating: True
Stack trace: at
System.Threading.ReaderWriterLock.ReleaseWriterLockInternal()
at Microsoft.SharePoint.Publishing.ThreadSafeCache`2.ReleaseWriterLock()
at Microsoft.SharePoint.Publishing.AclCache.LoadAllAcls(SPSite site)
at
Microsoft.SharePoint.Publishing.PublishingHttpModule.EnableCachingIfAppropriate(Obje
ct source, EventArgs e)
at
System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecut
ionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean&
completedSynchronously)

This has been traced back to an issue with the amount of ACLs across a site collection.  If you have more then 10,000 or more per site collection, and enable output caching, this issue may exhibit itself.  So, the question is, how can you verify the ACL count?  Here’s a quick SQL query you can run against the content database to show the number of ACLs per level:

--this query will select the number of ACLs used by any item with unique permissions

select ra.siteid as 'Site Collection GUID', p.scopeurl as 'Item URL', count(*) as 'Number of ACLs'
from roleassignment ra
join perms p
on ra.scopeid = p.scopeid
group by ra.siteid, p.scopeurl
order by ra.siteid

In the column marked Number of ACLs, you will see the count output.  This can be used to show how to best split out your site collection.  To work around this, you have two options:

1. Disable Output Caching for the entire site collection experiencing the
issue.

2. Re-architect the site collection by splitting the site collection into
separate site collections. The new design must result in the new site collections
having fewer than 10,000 ACLs, which is shown using the above query.

For more information on capacity planning, please refer to the following article:

Plan for software boundaries (Office SharePoint Server)
https://technet.microsoft.com/en-us/library/cc262787.aspx