SharePoint 2010 Search 'Dogfood' Part 2 - Query Health Reports

  • Article

One of my favorite features of SharePoint Server 2010 Search is the health reporting that we added.  With the query health reports you can very easily see what is going on with end user query latencies as well as pinpoint were the time is getting spent.  The capacity planning document talks about a lot of different guidelines around optimizing the query path, which I plan to detail in future posts, but the main thing to stress is that you should let the reports be your guide as to where to start investigating.  If the health reports indicate that query latencies are meeting your goals there is no real reason to proceed further with a performance analysis.  Although one benefit of doing an analysis of a healthy system is to provide an understanding of how close you are to capacities so you can plan for future growth.

 

Out of the box reports

There are two reports titled SharePoint Backend Query Latency & Overall Query Latency.  These two reports enable you to diagnose where the costly portion of the query is being executed.  The backend report (SharePoint Backend Query Latency) allows you to determine if the time is being spent within the Index (Full-text Query) or the Property store.  Moving up the call stack you have the front end (Overall Query Latency) report, with this report you can determine if there is an issue in rendering the UI.  For more details about each data series within the reports see the TechNet article Use search administration reports.  

 

SharePoint Backend Query Latency - This report typically has the largest amount of time being spent within the Full-text Index as this is the most time consuming portion of the query.  

 

Overall Query Latency - This report adds Server Rendering and Object Model to the above report (Backend.)  All of the time reported in red below is equivalent to what you would see in the backend report.

 

Query Latency Trend report

We also ship a third query health report.  This report is disabled out of the box as it requires additional data to be logged into the Web analytics and health data collection database.  This is also disabled under the premise that not all farms, especially content farms, will benefit from having this additional data, but if you are running a Search centric farm I would encourage you to enable this extra data collection.  Our SearchBeta has run with this extra data collection enabled all the time and there is no perceived impact to the performance of the system.  Just be sure that you segregate the Usage database away from the query system so the additional IO generated does not impact queries.

 

To enable the extra data collection use the following PowerShell commands:

$app = Get-SPEnterpriseSearchServiceApplication "Your application name"

$app = Set-SPEnterpriseSearchServiceApplication –VerboseQueryMonitoring "True"

 

Query Latency Trend - The purpose of this report is to provide an all up summary of how your query latencies are doing.  The percentiles include time spent in both the front end and backend.   The report also includes the system crawl rate to help correlate any trends in query latencies with what is going on in the crawl system. 

 

Utilizing the reports to direct diagnostics

The general process for diagnosing query health is to:

  • look first at the trending report (Query Latency Trend) to get an idea about how your latencies are trending throughout the day. Do you have a lot of peaks and valleys in the trends or just overall high latencies? If you have peaks and valleys you can look for trends in the peaks over time as well as narrow the time window to focus on a specific peak. 
  • Secondly you would go to either the front end (Overall Query Latency) or backend (SharePoint Backend Query Latency) report to see precisely where all of the time is being spent. Given this you know where to focus further diagnostic efforts: front end, Query Servers or the Property store.   

 

Dan Blood

Senior Test Engineer

Microsoft Corp