Set the Crawl Component to use a dedicated web front end (WFE) for crawling in SharePoint 2010

In SharePoint 2007 it was fairly easy to set the indexer to crawl a dedicated WFE.  In fact it was in the UI and very easy to change.  When you selected a dedicated indexer in SharePoint we would add an entry to the host file on the indexer.  This solution worked but was not very elegant.  It was also prone to errors since frequently a host file is modified in many common troubleshooting scenarios. 

In Microsoft SharePoint 2010 this functionality is different.  The most notable change is that it is no longer controlled in the UI.  Next, it no longer uses the host file.  You now provide a URL or multiple URLs that point to the server you want to crawl for the content.  To make the change you must use the SharePoint 2010 Management Shell.

In order to to make this work you need to extend a Web Application to a URL that will resolve to a single server.  So for instance lets say your main site is located at https://intranet.contoso.com.  You will need to extend the site to a new URL and port like https://WFE1:8080 or you can set it to the indexer itself https://INDEXER1:8080.  You could also very easily modify the script below to specify multiple dedicated servers which was not possible in SharePoint 2007.

Once you have extended the site you are ready to run the script. 

Note: Please be advised that you cannot use an SSL or host header site as the dedicated server.  

$webAppUrl = "https://www.testsite.com"                  ## Add the FQDN to your Default Zone URL
$URLOfDedicatedMachine = "https://wfe2:8080"            ## Add the URL of the dedicated WFE

$listOfUri = new-object System.Collections.Generic.List[System.Uri](1)
$zoneUrl = [Microsoft.SharePoint.Administration.SPUrlZone]'Default'

$webApp = Get-SPWebApplication –Identity $webAppUrl
$listOfUri.Add($URLOfDedicatedMachine);

if ($listOfUri[0].Scheme-eq 'http')
{
               $webApp.SiteDataServers.Remove($zoneUrl)         ## By default this has no items to remove
               $webApp.SiteDataServers.Add($zoneUrl,$listOfUri);
               $WebApp.Update()
}
elseif ($listOfUri[0].Scheme-eq 'https')
{
               echo "$listOfUri is an HTTPS site and cannot be used. You must extend the site you are trying to crawl to a non SSL web application. No changes have been made!"
}