Thoughts on Debugging IIS7


Hmm… it seems that ever since I made the blog entry about how to install PHP on IIS7, I seem to have become the default tech-support for PHP/IIS7 issues. That’ll teach me to ever post beta guidance again – I just get more “help, I’ve fallen and I can’t get up” questions, especially the sort when the user ignores the directions or thinks that their “modifications” to my instructions are just as good. Sigh… 😉


I mean, is it too much to expect that if you want to modify the configuration files by hand or commandline that you should have write access to the configuration files? The IIS7 UI automatically asks you for privilege elevation, but when you go on the commandline you are obviously on your own. Or that if I give instructions for configuring the PHP ISAPI handler to run using the ISAPI Module that you merely change the handler to run the PHP CGI EXE handler with the ISAPI Module (and not the CGI Module)?


However, this also makes me realize that if you think IIS6 is confusing and difficult to troubleshoot, IIS7 is even more open-ended and hence harder to troubleshoot. You really have to know how IIS7 operates and what you are configuring (and why) when you make IIS7 configuration changes… because one wrong step and the server will stop working, by-design due to its open extensibility.


For example, when it comes to requests not resulting in the proper response, my first question is “what handler generated the response – static file handler or arbitrary dynamic file handler” – because that determines the expected response behavior. However, it is clearly not how users think – users think of it as a file to download, HTML page with a CSS stylesheet, or ASP.Net web application not working. Unfortunately, given how configurable (and misconfigurable) IIS7 can be, using the user’s view of the issue will not be sufficient to troubleshoot, and we cannot make it any better/easier – there are so many things to misconfigure!


All it means is that if you customize beyond the default set of supported IIS7 configurations, you will pretty much be on your own. We try our best to make many module combinations work, but it will always be possible to configure IIS7 modules into a non-functional state.


For example:



  • If you [accidentally] mangle the ordering of the StaticFileModule, DirectoryListingModule, and DefaultDocumentModule modules, IIS7 will either refuse to serve default documents (regardless of how you enabled/configured it), refuse to do 301 courtesy-redirect, or refuse to show a directory listing. If you just stare at the IIS7 configuration, you will tear your hair out trying to figure it out… and FREB or ETW tracing won’t really help, either. You literally have to know how to order the modules to end up with expected IIS behavior, and requiring that knowledge is by-design. Similar dependencies exist for authentication and caching…

  • If you want to configure a “wildcard application mapping”, you have to chant the magic incantation of a perfect handler configuration. I’ll leave the exercise to the reader to come up with the spell. :-)

  • If you configure a handler AFTER the catch-all StaticFileHandler, it will NOT apply to any requests. If you just stare at the IIS7 configuration wondering why it is not running – it is there in the configuration section – you will go crazy unless you know how the handler section works.

  • If you configure an ISAPI DLL as a globalModule, reference a non-existing managed type as a module, or remove a referenced module from loading… good luck on the spotting the error! 😉

In short, if you run IIS7 in non-default fashion, it will require either perfect instructions or the user has to really understand IIS7 processing and behavior. Maybe there is a way to lower the requirements of running/using IIS7, but I have not come up with any. What are your thoughts?


//David

Comments (20)

  1. Kirit says:

    David, I’ve got to say this sounds like a nightmare. On IIS 6 I don’t need to change the application settings, but I do need to change some of the web site configuration – an ISAPI filter needs to run before there’s any chance of script pages for examples.

    Am I really going to have to learn all of this stuff in order to just get my applications working?

    Now I guess you’re going to say "You’re developing an IIS application using ISAPI. Of course you need to know this stuff, Duh!" and I guess that’s fair up to a point.

    I’ve been working on every version of IIS since the first one came out (ASP was so much better than writing plug-in DLLs for Compuserve’s web server) and I have to say that I’m very pleased to be able to treat it as a black box. I still don’t know what the metabase is on IIS 6 or why I’d want to do anything with it. I have other fish to fry and things to do. So long as IIS isn’t bothering me I promise not to bother it.

    Am I going to have a complete nightmare on IIS 7?

  2. TristanK says:

    I’ve had the same concerns.

    Using your examples, there are two things we can do:

    – an automated scan tool that looks for common misconfigurations (I know how much you love automating things) and warbles about them: "hey! Your StaticFileModule, DirectoryListingModule, and DefaultDocumentModules look backwards! If you’re not sure you wanted that, do you want me to fix?".

    You picked a rather well-formed example for that one; it’s the gray area of "I think I want that" that’s most concerning and frequently leads to problems…

    – a BLAT.EXE that renames the current config and blats the default one back in. I’d suggest including a "baseline" XML file, but there’s this tendency for people to move and modify rather than copy and rename, and then that’s the one easy fix gone!

  3. DaMour says:

    >> it will require either perfect instructions or the user has to really understand IIS7 processing and behavior

    Where does the end developer get information so that they can "really understand" IIS7? How much of the inner workings are you actually able to reveal?

  4. Ed Goward says:

    Eek, that does sound scary.

    In IIS5 and 6 there doesn’t seem to be an obvious "write a very verbose log file here" option like you get with ODBC drivers and other services.  If you could see a list of the handlers that were considered for each request and the one that generated the response then it would make life a lot easier.

    Maybe the information is there and I’m just blind – if it was more obvious in IIS7 it would be good.

  5. Justin says:

    Is microsoft thinking about creating IIS 7 training? There is very little good IIS 6 training information out there in my opinion. And i think it would also be beneficial to add IIS to the MCSE track again.

  6. David  TAG: You’ve been elected to do a webcast on IIS7 and setting it up for php and static files.  

    There is obviously a community of superdudes who have been salivating int he corner over IIS7, and who share your passion and want to dive-in head first.  If there is no water in the pool, maybe it’s time we put the water in?

    The "Its by design, I’m sorry you were a victim of gravity" just isnt going to cut it anymore.   :)

    I for one dev ASP.NET all day, with a pretty good understanding of IIS6, handlers, rewrites, modules etc. and I gotta say, the first time through even I f-d it up pretty bad.   If this is what we are going to roll out into release with then we better get some developer champions on the same level.  

    A: I’ll be the first to sign up for the webcast. :)

  7. David.Wang says:

    Kirit – If you use the same configuration code from IIS6, it will more than likely work without modification on IIS7, so no worries.

    While the IIS7 UI is very different from the prior MMC Snapin-based UI, I think it is a vast improvement and covers most of the common user usage cases.

    The biggest confusion that I am pointing out is around how to properly configure the guts of IIS7, the modules and handlers running on top of the new IIS7 Request Pipeline to provide IIS functionality. Fortunately, few people should need to do this on their own.

    For the most part, you don’t have to worry because we will preconfigure IIS7 with the proper modules and ordering to support IIS behavior, so you only worry if you are running custom CGI/ISAPI to modify IIS behavior. But even in those cases, if you use the original IIS configuration APIs, things should work out. It is when you choose to manipulate IIS7 modules/handlers configuration directly WITHOUT knowing what you are doing that you run into the trouble that I describe.

    Since IIS7 is openly and completely configurable, it is that much easier to misconfigure it to malfunction. However, IIS7 also comes with very detailed diagnostic/troubleshooting modes to help deal with the complexity – way better than any prior IIS version.

    //David

  8. David.Wang says:

    TristanK – Unfortunately, we can’t even save the "baseline" config.

    We planned to save a backup of the "baseline" config at the end of IIS installation so that you can easily restore it to the original "factory configuration", but we ran into one little problem… IIS7 is completely componentized into 40+ individually installable/configurable components, which adds this wrinkle:

    What components make up the "baseline"? Is it the 29 components you installed? or the 37 components that someone else selects?

    To make it harder, when you choose to install IIS7 components, Windows actually calls into IIS setup once for each component, so we do not know which "batch" of components constitute an installation.

    i.e. If you select 37 components of IIS7, IIS setup code actually gets invoked by Windows 37 times to install each component, instead of once to install 37 components, which looks no different than if you installed 29 components at once and later added 8 components – so IIS Setup does not know when it is "done" installing IIS and hence does not know when to take a baseline snapshot.

    Bottom line: you can save yourself a lot of hassle by making a backup of the %WINDIR%System32inetsrvconfig directory when you finish installing IIS7.

    //David

  9. David.Wang says:

    DaMour – I am going to try to reveal as much as you are interested in knowing. You just have to put effort into focusing your questions beyond the ambiguous "how does it all work". :-)

    I plan to write about IIS7 "Internals" just as I do about IIS6 "Internals". Directly from me to you with no obfuscation in-between.

    //David

  10. David.Wang says:

    Ed Goward – IIS7 has rich ETW Tracing and FREB which should satisfy what you are asking for. If you have not seen our TechEd presentations on advancements in IIS7 Troubleshooting, you should – I bet you will be salivating after seeing how it eases diagnosing many classes of IIS issues.

    For every failed request (where you define "failure" as some set of HTTP Status code), you can instruct IIS7 to automatically capture a trace of the entire pipeline execution and watch which CGI, ISAPI, Module, and Handler did what in the IIS7 Pipeline. And with ETW Tracing, you can trace requests from HTTP.SYS to IIS7 into ASP, ASP.Net, COM+, SQL, etc.

    In other words, you can enable FREB to trigger on 401s or 500s, and IIS7 will produce you a correlated log showing only 401s and 500s, exactly which module triggered the error code, and if you trace enough subsystems, you can probably get a good idea what caused each issue. All without needing a debugger and requiring just basic systems understanding.

    IIS7 actually allows you to correlate the various Windows components and subsystems that interacted on a given request. IIS6 has the basics of the same infrastructure so it is also pretty diagnosable; it is just noisier.

    So, eh, we do not write a verbose log file because it is rather simple-minded. We already do something better. :-)

    //David

  11. David.Wang says:

    Justin – Honestly, I have no idea. The product team has no knowledge nor involvement with people creating training for its product.

    //David

  12. David.Wang says:

    Eric – sure. :-)

    There are lots of community plans around IIS7. I will hopefully be involved with many of them in some way… because you know that I will try to give as much useful information, details, and assistance as possible.

    I say "hopefully" because unfortunately, my "job description" makes this outside my core responsibilities and my boss does not value me for doing things like this. He rather have me do what he wants, instead of what helps the IIS user or product.

    //David

  13. Nathan says:

    One of the things I would like to see is the ability to connect to an existing II6 web server and be able to import that into IIS 7 . I know this takes all the fun out of learning(cough  cough)a new direction , but it would be very beneficial .

  14. Thomas says:

    I must say, the IIS7 UI may be great if you are familiar with how precisely IIS7 works internally, but as someone coming from IIS6, it is a jumbled mess.

    I am trying to complete the simple task of getting .cgi files to execute using ISAPI Perl. This was dead-simple in IIS6 and previous versions.

    In IIS7 there are a variety of areas to explore for this addition: "CGI", "Handler Mappings," "ISAPI and CGI Restrictions," "ISAPI filters," and "Modules." They all seem to be somewhat related either to CGI or ISAPI, so it’s very confusing as to the proper procedure for simply adding a new CGI handler and making it work for all *.cgi files.

    The way it’s currently configured, I receive this error:

    HTTP Error 500.0 – Internal Server Error

    Description: Handler "Perl .cgi" has a bad module "IsapiModule" in its module list

    Error Code: 0x8007000d

    Notification: ExecuteRequestHandler

    Module: IIS Web Core

    Requested URL: http://test:80/test.p

    Physical Path: D:testwwwtest.p

    Logon User: Anonymous

    Logon Method: Anonymous

    Handler: Perl .cgi

    *****

    Is this really the best interface the UI team can come up with for IIS?

  15. David.Wang says:

    Thomas – I agree that the IIS7 UI is quite different from all prior IIS versions.

    Personally I am not a UI person (I directly create and tweak the XML in the necessary IIS7 .config files) so you can take my opinion with a heavy dose of salt.

    However, I like the IIS7 UI precisely because it organizes IIS settings so that as one learns more about IIS, the settings become easier to reason and find.

    The IIS UI team spent a *lot* of time on the UI, both in usability by both novices and veterans, and there were a *lot* of different and often conflicting requirements from different user segments. The problem is that what is best for the novice will not work for the developer, nor will it work for the IT professional, and none will suffice for the hosted scenario. There is no way to craft 4 different UIs…

    The prior IIS UI was nothing more than a glorified metabase editor. And people have gotten used to magically navigating to particular tabs and checkboxes not because it made logical sense but because they just remember the navigation pattern to do something they wanted. For example, the "Home Directory"/"Virtual Directory" with the random "Create" application button that do nothing obvious and has three radio buttons that add/remove text with radically different functionality and syntax. Does any of that make sense? Or have you just gotten used to how it works?

    So, while I commiserate with your troubles in accomplishing what you want to do, I ultimately believe the growing pains is worth it. By the time one is adding handlers/modules, they MUST understand the terminology involved. If you don’t understand the terminology, then you should not be adding handlers/modules and you should blame the product for lacking a setup program to take care of these details.

    //David

  16. jay says:

    Server Error in Application "Default Web Site"

    ——————————————————————————–

    HTTP Error 404.3 – Not Found

    Description: The page you are requesting cannot be served because of the Multipurpose Internet Mail Extensions (MIME) map policy that is configured on the Web server. The page you requested has a file name extension that is not recognized, and is not allowed.

    Error Code: 0x80070032

    Notification: ExecuteRequestHandler

    Module: StaticFileModule

    Requested URL: http://localhost/main_login.php

    Physical Path: C:inetpubwwwrootmain_login.php

    Logon User: Anonymous

    Logon Method: Anonymous

    Failed Request Tracing Log Directory: C:inetpubwwwroot

    Handler: StaticFile

  17. jay says:

    Server Error in Application "Default Web Site"

    ——————————————————————————–

    HTTP Error 404.3 – Not Found

    Description: The page you are requesting cannot be served because of the Multipurpose Internet Mail Extensions (MIME) map policy that is configured on the Web server. The page you requested has a file name extension that is not recognized, and is not allowed.

    Error Code: 0x80070032

    Notification: ExecuteRequestHandler

    Module: StaticFileModule

    Requested URL: http://localhost/main_login.php

    Physical Path: C:inetpubwwwrootmain_login.php

    Logon User: Anonymous

    Logon Method: Anonymous

    Failed Request Tracing Log Directory: C:inetpubwwwroot

    Handler: StaticFile

  18. David.Wang says:

    jay – you need to add a handler for PHP and enable the CGI or ISAPI handler you configure. See my associated blog entry on how to install PHP on clean-installed IIS7 for example.

    //David