The Case of the Disappearing Hosts File


Hmmm...how should I phrase this?


It has been a very educational couple of weeks on my current SharePoint project.


During the rebuild of our Test environment, the SharePoint Products and Technologies Configuration Wizard failed when it was unable to find the %SystemRoot%\System32\drivers\etc\hosts file. We had encountered this error during the original installation on the SSP server in the Test environment because someone had renamed the file from hosts to hosts-old. Therefore we suspected the same problem this time around, thinking that perhaps there was some scheduled script or group policy that was disabling local host name resolution.


For those of you that may not have attempted to read the status messages as they flash by in the Configuration Wizard, step 4 changes the permissions on the hosts file to grant the WSS_ADMIN_WPG group the following permissions:



  • List Folder / Read Data

  • Read Attributes

  • Read Extended Attributes

  • Create Files / Write Data

  • Create Folders / Append Data

  • Write Attributes

  • Write Extended Attributes

  • Delete

  • Read Permissions

After recreating the hosts file, I was able to successfully complete the Configuration Wizard (because the security settings on the hosts files could now be set in step 4). However, a short time later, I noticed the following in the event log:


Application Server Administration job failed for service instance
Microsoft.Office.Server.Search.Administration.SearchServiceInstance (...).

Reason: Could not find file 'D:\WINNT\system32\drivers\etc\HOSTS'.

Techinal Support Details:
System.IO.FileNotFoundException: Could not find file 'D:\WINNT\system32\drivers\etc\HOSTS'.
File name: 'D:\WINNT\system32\drivers\etc\HOSTS'
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(...)
at System.IO.FileStream..ctor(...)
at System.IO.StreamReader..ctor(...)
at System.IO.FileInfo.OpenText()
at Microsoft.Search.Administration.Security.HOSTSFile.ParseHOSTSFile(...)
at Microsoft.Search.Administration.Security.HOSTSFile.ConfigureDedicatedGathering(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.SynchronizeDefaultContentSource(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(...)


Argh! The hosts file has disappeared again!


I restored the hosts file again and set the permissions manually for the WSS_ADMIN_WPG group. However I noticed that it quickly disappeared again.


I then restored the hosts file (yet) again, but did not give the WSS_ADMIN_WPG group permission to delete the file. This resulted in the following event log entry:


Application Server Administration job failed for service instance
Microsoft.Office.Server.Search.Administration.SearchServiceInstance (...).

Reason: Access to the path 'D:\WINNT\system32\drivers\etc\HOSTS' is denied.

Techinal Support Details:
System.UnauthorizedAccessException: Access to the path 'D:\WINNT\system32\drivers\etc\HOSTS' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileInfo.Delete()
at Microsoft.Search.Administration.Security.HOSTSFile.CleanupDedicatedGathering(...)
at Microsoft.Search.Administration.Security.HOSTSFile.ConfigureDedicatedGathering(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.SynchronizeDefaultContentSource(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(...)


Aha! So apparently the guilty party for deleting my hosts file isn't some malicious system administrator or group policy setting, but rather it is the Windows SharePoint Services Timer itself!


It turns out that this is a bug in MOSS 2007 (although I am still waiting for PSS to formally acknowledge that this is a bug). Nevertheless, I am convinced that this is a bug. Here's why:


I never recommend to customers having your service accounts be members of the local Administrators group. Quite honestly, this is simply too dangerous and presents a whole slew of risks that are beyond the scope of this posting. Since the service account that I am using is not a member of the local Administrators group, when the timer job deletes the file, it does not have permission to recreate the file. Recall that earlier I mentioned that step 4 of the Configuration Wizard only grants permissions on the hosts file itself to the WSS_ADMIN_WPG group (which the service account is a member of). Hence the disappearing hosts file.


The workaround is to grant the following permissions for the WSS_ADMIN_WPG on the %SystemRoot%\System32\drivers\etc folder:



  • Traverse Folder / Execute File

  • List Folder / Read Data

  • Read Attributes

  • Read Extended Attributes

  • Create Files / Write Data

  • Read Permissions

For those of you that may be wondering why SharePoint needs access to the hosts file at all, the answer is due to one of the configuration settings that you can specify for Office SharePoint Server Search.


In our case, we are using a farm configuration similar to what used to be referred in SPS 2003 as a "Medium farm" -- i.e. two front-end Web servers and one SSP server (previously referred to as the job/index server). In order to minimize the impact on users, we have the index server (i.e. the SSP) crawl itself.


In Central Administration, if you specify a dedicated front-end for crawling contant, then a timer job is created to add an entry to your hosts file to force the Web application (i.e. the host header) to resolve to that server. Unfortunately, instead of "editing" the file in-place, the developer who implemented this feature decided it would be easier to just read the hosts file, add the appropriate entries, delete the original file, and then create a new file. If your SharePoint farm account (i.e. the one that the Windows SharePoint Servers Timer runs under) is a member of the local Administrators group then there is no problem (which appears to be how this feature was tested). However, if you adhere to the "principle of least privilege" then there's  definitely a problem.


Unfortunately we did not discover the bug with the disappearing hosts file in DEV because we are using a single server configuration there (and therefore did not specify a dedicated Web front-end for crawling).


I'll be writing a few more blogs posts this weekend to describe a couple of the other "gotchas" I have discovered in the last couple of weeks, but right now it is time for some pancakes!

Comments (13)
  1. maclau says:

    Hey there!

    I have followed your instructions and i still get the error every minute abou the access denied to he host file… any news on this?

  2. Can you post the exact CACLS/XCACLS command?

    Obviously,

     CACLS %SystemRoot%System32driversetc /E /G WSS_ADMIN_WPG:C

    Will work, but that also grants more permissions then stated…

  3. Even with MOSS SP1 and the December 2008 CU, I still find myself having to refer to this blog post each and every time I build up a new MOSS environment.

    Note that in Windows Server 2008, you’ll first need to take ownership of the %SystemRoot%System32driversetchosts file (from TrustedInstaller) before you can change the permissions.

    As for maclau’s comment, I’m not sure why you would still be getting the errors if you followed these instructions explicitly. I’ve used this hack no less than 20 times since I originally discovered the problem. Once I make the changes, the HOSTS file magically reappears within a minute (the frequency of the SharePoint Timer job that nukes and recreates the hosts file).

    As for Christopher’s comment, I don’t use CACLS.exe, but rather do it interactively each time. If you find this too cumbersome, I’ll defer the translation to CACLS as an exercise for the reader 😉

  4. The Nihl says:

    Potential Issues Installing SharePoint 2007 Service Pack 2

  5. Great post – thanks for the detailed explanation!

  6. James says:

    I had the similar issue with an additional problem. I have MOSS 2007 and SQL 2008 installed on separate Windows Server 2008 machines and a DNS Alias for the MOSS. When I configured Alternate Mappings for DNS Alias, MOSS added a new entry to the HOSTS file and it started causing authentication problems. After struggling a while to understand the root cause, I removed the line and made the HOSTS file read-only. Until then MOSS has been adding errors to the event log, indicating its inability to update the HOSTS file. I would appreciate a better solution to this problem.

  7. @James,

    Without knowing more details about your specific issue, it sounds like it might be the problem described in KB 896861, and also mentioned in the following post:

    http://blogs.msdn.com/jjameson/archive/2009/02/10/issues-with-running-moss-2007-on-windows-server-2008.aspx

    Hope that helps.

  8. Pete says:

    This did not resolve my issue either.  Is a reboort or stop and start of services required?

  9. Nope…I’ve never had to reboot or restart services.

    I recommend you call Microsoft Support. It’s a bug — you shouldn’t be charged for the support incident.

  10. I should clarify…

    If your problem is caused by a missing hosts file, then once you fix the permissions you shoudn’t need to reboot. (The next time the SharePoint timer job runs, it will successfully write out the updated hosts file.)

    If, on the other hand, your problem is caused by the "loopback" issues described in my other post, then yes, you definitely need to reboot after applying the fix described in the KB article.

  11. Eddi Wabnitz says:

    Hi,

    placing the proper rights on the folder as described ….

    It didn´t solve my problem either ….. but then I renamed my own replaced hosts file. The timer job then created a HOSTS file on it´s own, so it did solve my problem actually ….. The only proble was that the timer job had to create the file itself …

  12. Eddi Wabnitz says:

    Hi,

    placing the proper rights on the folder as described ….

    It didn´t solve my problem either ….. but then I renamed my own replaced hosts file. The timer job then created a HOSTS file on it´s own, so it did solve my problem actually ….. The only proble was that the timer job had to create the file itself …

  13. Mark says:

    I just encountered the disappearing hosts file issue.  However my issue was not around permissions.

    The behavior I noticed is that the hosts file was moved and renamed to the ULS Logs folder by the Farm account.  

    Did anyone observe this behavior too ?  thanks a lot for your post.  mark

Comments are closed.

Skip to main content