SharePoint Indexing Limited by 64 kB ACL Limit

I want to share a problem I had a few month ago when we were indexing the the items in a SharePoint-based application.

Initial problem

When crawling one of the sites the engine stopped indexing and logged the error "The parameter is incorrect". Which parameter? Why is it incorrect?

Analysis

After intensive discussion and testing we managed to relate the problem to the number of users who had access to the site. Sites with few members were indexed just fine while sites with 2000+ members failed. We did emperical tests and found that in our particular environment the index engine failed if we had more than 1812 members.

What was reallt interesting was that when we were just above the 1812-limit a new error message was displayed: "Search cannot crawl the item, because its Access Control List exceeded 64 KB". Note that this error message was only shown when we exceeded the limit with a small margin, when adding additional members the previous error ocurred again.

Now that SharePoint gave us the proper error we were able to understand what was happening, the 64 kB limit for an ACL is a Windows-limit, not something SharePoint enforces. With this information I was also able to find a SharePoint 2003 KB article which described the same problem in the previous version. You can also see the error code at https://msdn2.microsoft.com/en-us/library/aa981067.aspx, look for PRTH_E_ACL_TOO_BIG.

The limit of 1812 member will not be your limitation, if will probably be something else in the span 1500-2000 members.

Workaround

Our solution was to require sites with over 1500 members to use an AD group for keeping track of membership, as recommended by the KB for SPS 2003. One disadvantage with this approach is that you must maintain membership by updating the AD group so you can no longer request membership using SharePoint standard dialogs.

As a side note, why did we have so many members? We were building a Knowledge Management tool and some of the communities had a large number of member.