HTTP Error 503. The service is unavailable.
One fine day, I thought of looking at new OOB modules in IIS and thought of first making sure that my IIS normal pages are working fine or not. We ahead and tried browsing to normal iisstart.htm page under default website. To my surprise I found the following error when browsing to iisstart.htm page.
Service Unavailable
--------------------------------------------------------------------------------
HTTP Error 503. The service is unavailable.
-- Following is the snap-shot of the error message.
Since I knew for sure that requests are not even reaching IIS server, so went ahead and looked in the HTTPErr logfile (found in location C:\Windows\System32\LogFiles). Found the following entries in it.
#Version: 1.0
#Date: 2009-07-24 02:05:29
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri sc-status s-siteid s-reason s-queuename
2009-07-26 13:22:57 ::1%0 49161 ::1%0 80 HTTP/1.1 GET /iisstart.htm 503 1 AppOffline DefaultAppPool
2009-07-26 13:24:36 ::1%0 49162 ::1%0 80 HTTP/1.1 GET /iisstart.htm 503 1 AppOffline DefaultAppPool
2009-07-26 13:27:27 ::1%0 49163 ::1%0 80 HTTP/1.1 GET /iisstart.htm 503 1 AppOffline DefaultAppPool
2009-07-26 13:29:29 ::1%0 49165 ::1%0 80 HTTP/1.1 GET /time.asp 503 1 AppOffline DefaultAppPool
So we could conclude that Application Pool was getting offline, but the reason for it was not clear. Decided to look into the event viewer, as the application pool was getting disabled, IIS would for sure log it in event viewer. Checked in the System event logs and found the following
Log Name: System
Source: Microsoft-Windows-WAS
Date: 26-07-2009 06:29:29
Event ID: 5002
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: COMP2008
Description:
Application pool 'DefaultAppPool' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
Event Xml:
<Event xmlns="https://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
<EventID Qualifiers="49152">5002</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2009-07-26T13:29:29.000Z" />
<EventRecordID>20838</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>System</Channel>
<Computer>COMP2008</Computer>
<Security />
</System>
<EventData>
<Data Name="AppPoolID">DefaultAppPool</Data>
<Binary>
</Binary>
</EventData>
</Event>
Log Name: System
Source: Microsoft-Windows-WAS
Date: 26-07-2009 06:29:29
Event ID: 5139
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: COMP2008
Description:
A listener channel for protocol 'http' in worker process '2252' serving application pool 'DefaultAppPool' reported a listener channel failure. The data field contains the error number.
Event Xml:
<Event xmlns="https://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
<EventID Qualifiers="32768">5139</EventID>
<Version>0</Version>
<Level>3</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2009-07-26T13:29:29.000Z" />
<EventRecordID>20837</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>System</Channel>
<Computer>COMP2008</Computer>
<Security />
</System>
<EventData>
<Data Name="AppPoolID">DefaultAppPool</Data>
<Data Name="ProcessID">2252</Data>
<Data Name="param3">0</Data>
<Data Name="ProtocolID">http</Data>
<Binary>05400080</Binary>
</EventData>
</Event>
Although thought that there would not be anything logged in Application event viewer, to my surprise I found something interesting which helped me in resolving the issue..
Log Name: Application
Source: Microsoft-Windows-IIS-W3SVC-WP
Date: 28-07-2009 19:05:54
Event ID: 2293
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: COMP2008
Description:
The Module name FastcgiTesting-Register path C:\Windows\System32\inetsrv\iisfcgi.dll returned an error from registration. The data is the error.
Event Xml:
<Event xmlns="https://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-IIS-W3SVC-WP" Guid="{670080D9-742A-4187-8D16-41143D1290BD}" EventSourceName="W3SVC-WP" />
<EventID Qualifiers="49152">2293</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2009-07-29T02:05:54.000Z" />
<EventRecordID>1163</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>Application</Channel>
<Computer>COMP2008</Computer>
<Security />
</System>
<EventData>
<Data Name="ModuleDll">FastcgiTesting-Register</Data>
<Data Name="Path">C:\Windows\System32\inetsrv\iisfcgi.dll</Data>
<Binary>05400080</Binary>
</EventData>
</Event>
Log Name: Application
Source: Microsoft-Windows-IIS-W3SVC-WP
Date: 28-07-2009 19:05:45
Event ID: 2293
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: COMP2008
Description:
The Module name FastcgiTesting-Register path C:\Windows\System32\inetsrv\iisfcgi.dll returned an error from registration. The data is the error.
Event Xml:
<Event xmlns="https://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-IIS-W3SVC-WP" Guid="{670080D9-742A-4187-8D16-41143D1290BD}" EventSourceName="W3SVC-WP" />
<EventID Qualifiers="49152">2293</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2009-07-29T02:05:45.000Z" />
<EventRecordID>1161</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>Application</Channel>
<Computer>COMP2008</Computer>
<Security />
</System>
<EventData>
<Data Name="ModuleDll">FastcgiTesting-Register</Data>
<Data Name="Path">C:\Windows\System32\inetsrv\iisfcgi.dll</Data>
<Binary>05400080</Binary>
</EventData>
</Event>
So we could atleast now conclude that there was FastcgiTesting-Register Module, which tried registering itself and failed to register successfully. (Now I remembered that I was playing with FCGI Module some days back and somehow did not completely finish the module and left the incomplete module in IIS, without removing them). Since this registration was not successful, Application Pool was unable to start itself.
Went ahead and removed the mapping from Server Level. To do this we did the following
1. Highlighted the computer name in IIS Manager and under Features View -> Selected the Modules -> Right Click -> Open Feature.
2. Tried looking in for the name FastcgiTesting-Register Module.
3. Went ahead and selected that and clicked on Remove from Actions pane.
4. Did iisreset.
Now we thought that we had completely removed from IIS and now went ahead and tried browsing the iisstart.htm page and still the same error as above. In event logs still we were seeing the same errors. It had not gone. Checked all locations in IIS Manager and could not find any. Now finally decided to open up applicationHost.config file ( Found in location: C:\Windows\System32\inetsrv\config). Tried looking in for the keyword (FastcgiTesting-Register) and to my surprise I found one at <globalModules> section.
<globalModules>
<add name="FastcgiTesting-Register" image="C:\Windows\System32\inetsrv\iisfcgi.dll" preCondition="" />
</globalModules>
Went ahead and manually removed the line from applicationHost.config file and then saved them. (Note: Directly editing applicationHost.config file may cause other issues. So please make sure that it is edited carefully). Did iisreset and then tried to browse to iisstart.htm and this time it came up fine without any issues.
Learning:
Please make sure that whatever labs are performed on IIS server, please take a back-up of IIS and then once the lab is completed, please restore the working server configuration. This would save a lot time, which we ended up spending for the above issue.
With this I conclude, will share many more in future.
Until then bye :)
Vandana