I received the following question on the SPYam group on Yammer, and decided to share the information with the broader audience via my blog.
For SharePoint security Microsoft always recommended to use Active directory group inside SharePoint group. But this guidance now changed Microsoft now recommend:
"We do not recommend SharePoint groups to assign permissions to sites. When a SharePoint group is used to assign permissions, a full crawl of the index occurs. Instead, we recommend Active Directory Domain Services (AD DS) groups."
Can someone from Microsoft confirm this please?!
Happy to try
Configuring the Environment
To demonstrate the effect of adding a user to a SharePoint group, adding an Active Directory group, and adding users to an Active Directory group, I want to isolate any changes as much as possible from the rest of my environment. I configured my environment with a new web application CrawlDemo.Contoso.local, and a root site collection using the Team Site template. The site collection administrator is a single user, “contoso\administrator”.
In order to isolate the effect of search crawls to just this web application, I configured a new content source that contains my new web application’s URL.
This lets me isolate any crawls to just that one web application. The rest of the settings for the content source:
- Crawl Settings: Crawl everything under the hostname for each start address
- Crawl Schedules: Enable Incremental Crawls
- Content Source Priority: Normal
I do not have continuous crawls configured for this environment, and none of my content sources are currently scheduled, so I can be sure that crawl information will be isolated to this single web application.
Before continuing, I make sure that the crawl account has Full Read permission to the web application by checking the User Policy for the web application in Central Administration.
Note the text in red:
Adding or updating Web application policy with new users or groups will trigger a SharePoint Search crawl over all content covered by that policy. This can reduce search crawl freshness and increase crawl load. Consider using security groups at the policy level and add/remove users from security groups to avoid this.
We’ll come back to that later in this post as it is directly relevant to what we are discussing.
Once everything is configured, I initiate a full crawl of the content source. Reviewing the Crawl Health report in Central Administration for Crawl Rate, I see that all 37 items were crawled.
If we focus our attention on the Summary table to the right of the screen, we can see the total items crawled was 37 items.
This means that the full number of items crawled was 37 items. When we run an incremental crawl, we should only crawl the changed items and not every item. The rest of this post focuses on the impact that security changes have on crawling.
A Note on Performance
I am in no way indicating that my demonstrations in this post be used for any type of performance benchmarking. I am running all of these tests on a single laptop on a single drive using virtualized images that are not optimized for performance. Further, I have other processes running on the machine, so the numbers that I show do not represent a production environment. That said, we can make some general assumptions based on the data represented.
Understanding the EventCache
In each content database, there is a table named “EventCache” which contains a change log for everything in that content database. Once the crawl is complete, I query it using the following SQL statement:
select EventType, ObjectType, EventTime from EventCache with (nolock) where EventTime > '2013-05-06 16:37:36.750'
Note that it is unsupported to query SharePoint’s databases directly and should not be done in production without the direct involvement of a Microsoft support engineer. To minimize any potential effects of the query, I use the SQL NOLOCK hint to read uncommitted records and avoid introducing a table lock.
Initially this query returns zero results because the time stamp used in the query is the last time stamp from when I created the site collection. Let’s add a few documents to the Documents library in our new site.
We query the table again, and this time we see results in the EventCache table in the content database where the documents were added.
That makes complete sense, 4 documents added, 4 rows in the database. We initiate an incremental crawl (not a full crawl) and wait for it to complete. Once complete, click the Crawl Health Reports link in the search service application in Central Administration.
The important part to call out is the Summary table on the right of the screen that shows 12 items were crawled during this incremental crawl, none of which were security items.
When the search crawler tries to determine what changes were applied, it reads the information in the EventCache table to calculate what needs to be crawled. The thing to learn from this section is that there is a table, EventCache, which the crawler uses to determine changes from each of the content databases.
I run another incremental crawl, and the number of items decreases to 6 crawled items, and another decreases it to 5 items crawled. No new rows were in the EventCache table, so SharePoint is simply crawling the top level pages to determine if any content changes appear.
Effect of Adding a User to a SharePoint Group
Let’s add a single domain user to the “Crawl Demo Members” group.
We now run our SQL query and see that a row has been added to the EventCache table.
The new value, EventType 2097152, indicates a change where a member was added to a group (actually, 3 new rows were added, but I am only showing the relevant new row). Recall that the search crawler reads the information in the EventCache table to calculate what needs to be crawled. We run another incremental crawl and then inspect the health reports to see what happened. Notice the red bar that now shows, the legend for the chart indicates this is a “security only” crawl.
Because we added a user to the group, everything in the site needs to be crawled in order to re-calculate the ACLs for the site. Focusing on the Summary table at the right, we see that 36 security items were crawled simply because we added a user to a group.
We next turn to the crawl log to inspect a summary of our crawls so far.
Notice the value in the “Type” column has a letter “F” (indicating full crawl) or “I” (indicating incremental crawl). The time for the incremental crawl went from 2:30 to 2:10 to 1:40 before we added a user to the group. When we added the user to the group, it jumped up to 2:50 again because the ACLs for the items had to be recalculated. The entire index had to be crawled to calculate ACLs.
In a large SharePoint environment with many documents, adding a user to a SharePoint group can have a significant impact on crawl performance.
Effect of Adding an Active Directory Group to a SharePoint Group
We just saw what happens when adding Active Directory Domain Services users to a SharePoint group. Now let’s see the impact of adding an Active Directory group to a SharePoint group. The effect will be less educational than before because the results will look nearly identical to what we just saw. I create a new AD group, “MyCrawlGroup1” and add 12 users to the group in Active Directory.
Next, I add “MyCrawlGroup1” to the “Crawl Demo Members” SharePoint group.
We then go to the EventCache table to see what the changes were. I get 3 new rows, the most important one is highlighted.
As we saw before, this change indicates that a member was added to a group. We initiate an incremental crawl again, and just like before we see that we have the same number of security-only updates in the index. Notice the red bar again.
Focusing on the Summary table, we see that we again have 36 security items, the same number as when we added a single user to a SharePoint group.
If we check the crawl times, we can see that the time increased again. This time, the time increased because SharePoint had to calculate the ACLs for the members of the domain group. While the difference between the time spent calculating the members of the domain group versus adding a user directly is not statistically relevant (the difference of 10 seconds can be attributed to a number of factors), the fact that adding a domain group caused a security crawl is significant.
The point to understand in this section is that any time you add users or AD groups to SharePoint groups, your next incremental crawl causes the entire index to be crawled in order to calculate the ACLs.
Effect of Adding an Active Directory Group to a Site
This should be predictable by now. We will add an Active Directory group directly to a site instead of adding to a SharePoint group. This time, I create a group with 12 different users in it called “MyCrawlGroup2”.
Next, we will add the group directly to the site, granting this domain group Edit permissions just like the “Crawl Demo Members” group has. When I go into “Site settings / Site permissions” and click the “Grant Permissions” button, an interesting thing happens, highlighted below.
SharePoint defaults to adding the new user or AD group to an existing SharePoint group. Since we already proved what happens when you add a domain group to a SharePoint group, let’s change the drop-down to “Edit” instead of “Crawl Demo Members [Edit]”, meaning the group now is added to the site with the same level of permissions as the SharePoint group.
We query the EventCache table and highlight the interesting row.
EventType 524288 indicates “change assignment add”, and object type 4 means “Web”, so we are changing the permission on the web itself. As expected, we run the incremental crawl, and you can guess what happens… a security crawl again. Notice the red bar.
Inspecting the crawl log, we see that the time increased again. Hopefully not statistically relevant, but the time has increased a bit in our crawl logs.
This should come as no surprise, adding a security object directly to the site permissions will cause the entire index to be crawled to calculate the ACLs.
Effect of Adding Users to Active Directory Groups
By now, you’re hopefully wondering, where’s the payoff for trudging through this post and all the pictures and data so far? Well, you’re here! Let’s see what happens when I add 12 more users to the security group “MyCrawlGroup1”, testing if adding users to a domain group that is contained within a SharePoint group causes security crawls.
First, we query the EventCache table to see if any changes were recorded since the last record was written. Nope!
With no changes in the EventCache table, our new users will be able to access the site because we added them to the Active Directory group, but no changes were recorded to SharePoint to trigger a security crawl. We check the report, and see that the “Security only” red bar no longer shows in the graph.
Further, looking at the crawl log, our crawl time significantly decreased from 3:31 to just 1:50.
The lesson to learn is that adding users to an Active Directory group that already has permission in a SharePoint group does not trigger a full crawl of the index to calculate changes in permissions.
Now let’s try adding users to our “MyCrawlGroup2”, testing to see if adding users to an Active Directory group that already has permissions to the site causes a full crawl of the index. I add 12 users to the AD group.
Of course, there are no new records in SQL.
And our incremental crawl runs again. The time stays consistent with zero security updates.
The lesson here is that adding users to an Active Directory group that has already been granted permission to a SharePoint site directly does not initiate a full crawl of the index to calculate security changes.
Bonus Material: Adding a User Policy to the Web Application
By now, you’ve seen that adding a user or AD group to a SharePoint group or site causes a full index to calculate the permissions. At the beginning of this post, I called out the User Policy button in Central Administration, noting the text in red:
Adding or updating Web application policy with new users or groups will trigger a SharePoint Search crawl over all content covered by that policy.
Let’s see what happens when we try to add a user. In a previous post, I covered PowerShell that configures the object cache accounts. Configuring the object cache accounts requires that you add Full Read permission to the Portal Super Reader and Full Control to the Portal Super Reader account. I add the two accounts as part of the configuration steps necessary.
Next, I check SQL.
EventType 8192 means change or modify, and object type 2048 means Security Policy. As you can reasonably expect by now, this causes a full index to calculate permissions as described in the red text on that page. Notice the red bar is back.
The lesson to learn is that it is important that you configure the object cache accounts so they have permission to the cache (see my blog posting Setting Object Cache Accounts in SharePoint 2010, the same steps apply for SharePoint 2013) while understanding the impact that adding the permission can have on crawl performance. Adding a user policy to the web application causes a full index to calculate permissions of all objects covered by that policy. You still need to configure the object cache accounts, but make sure that you plan for the potential impact on crawl performance when you apply this change.
Hopefully this post shed some light on how security works with SharePoint and the effect that adding users or Active Directory groups can have on crawl performance. Adding users to Active Directory groups that have already been granted permissions to SharePoint does not affect crawl performance. Adding users to Active Directory groups is the recommendation for managing security permissions in SharePoint.