HTTP POST Fails for Anonymous Authentication


Question:


I have ASP that receives a POST, displays the parameters and performs additional work. Authentication was anonymous (IUSR_<machine>) and also Integrated Windows.


Then I turned off Integrated auth (we can discuss reasons for this later). I am still able to process the page, but evidently there is NO POST data. ASP code does run, I can see Response.Write output, but all POST params are empty.


I then changed the IIS authentication of anonymous user account to a local system user account (new user, new password), still no luck. Finally I tried changing to local Administrator, still no luck.


Logfiles give me no clue – IIS seems to be happy that it correctly processed the file.


Is there any hope for me?


Answer:


We are just talking about software; of course there is hope. Reinstallation is always an option. 🙂 It’s not like water, fire, bullets, or outer-space where you only get one chance and mistakes are a bit more costly…


I believe you are observing the behavior described in KB 251404. Classic mixed authentication conflict with browser-side optimization. Basically, whenever you POST with Internet Explorer to a URL that uses anonymous authentication AND the browser has visited other NTLM authenticated URLs from that website, you will encounter this behavior.


One of the most common way for people to run into this situation is with custom authentication. They usually enable custom authentication on all their webpages which redirect unauthenticated requests to a /logon.asp URL which uses anonymous authentication to present a FORM for users to login with. However, they find that on the POST, the username/password are NOT given to /logon.asp and the custom authentication fails.


The ways to fix this? Either:



  1. Disable the client-side optimization (Registry Key mentioned in the KB Article)
  2. Use an uniform authentication protocol on the server (which has other benefits – Secure Single Sign On pretty much requires uniform authentication protocol because any other mechanism must be insecure, as I detail why in this blog entry).

Why does this happen?


The underlying cause of this behavior is an optimization in Internet Explorer. Why does this apparently broken optimization exist?


Well, let’s first look at the classic POST request sequence to an authenticated URL:



  1. Client makes anonymous POST request with 1GB entity body to a URL which requires authentication on the server.
  2. Server rejects the POST with a 401 response and indicates with WWW-Authenticate: headers the authentication protocols the client to use.
  3. Client makes another POST request with 1 GB entity body to the URL in accordance to the authentication protocol, negotiates authentication (this may involve additional requests/responses, depending on protocol), and eventually the URL is executed.

The reason that the initial request is anonymous is because HTTP clients are not clairvoyant – they cannot know that the server requires authentication prior to making that first request – so making the initial request anonymously makes sense since the server will reject with a 401 response if authentication is required.


After the HTTP client receives the WWW-authenticate headers, it knows what authentication protocols are required by the server (at least, it thinks… this is where mixed authentication foils the optimization), so it can attempt some optimization techniques. For example:



  • If the client negotiated Basic authentication, it knows that every subsequent request must have the Authorization: header. Thus, step #1 is not needed and the client should just perform step #3 with both the entity body and the necessary Authorization: header.
  • If the client negotiated NTLM authentication, it knows that the second leg of the NTLM handshake during step #3 is a obligatory 401.1 response, so sending entity body for that leg of the handshake is not useful. Thus, a reasonable optimization is to NOT send any entity body until the final leg of the handshake, when the URL is finally executed.

These optimizations make a lot of sense because sending 1GB entity body during the authentication handshake is simply a waste of bandwidth, especially for a request that will be rejected as a part of the authentication protocol. In the old days of modems, the optimization can easily save hours off of an authenticated POST. In today’s broadband environment, it still shaves off minutes.


In fact, optimal websites today will even truncate their 401.1 and 401.2 custom error responses to further reduce bandwidth usage during anticipated authentication.


How does the Optimization fail?


Now, the astute reader should realize that this optimization fails for a server in a mixed authentication configuration. Here’s how it fails:



  • Suppose the HTTP Client encounters a WWW-Authenticate: NTLM header returned by the server during an earlier request. It now THINKS this server requires NTLM authentication.
  • Later on, the HTTP Client is told to POST a FORM to a URL on this server. Unknown to the client, this URL only requires Anonymous Authentication.
  • Since the HTTP Client thinks the server requires NTLM Authentication (that’s what the earlier request told it), it chooses to NOT send the entity body for what it anticipates to be the second leg of the NTLM handshake.
  • However, since the URL only requires Anonymous Authentication, no NTLM handshake ever takes place. Thus, the URL executes as-is… without the entity body.
  • Since the entity body contains all FORM parameters, this sequence of events effectively loses all of them when the URL executes

And why does it work when the URL has Integrated Authentication enabled? Well, the NTLM handshake now takes place and “consumes” this optimized request, and on the final leg of the handshake the entity body is dutifully POSTed by the HTTP Client… and since the handshake completes successfully, the URL executes and sees the POSTed entity body. Everything works.


//David

Comments (11)

  1. Mike A. says:

    Perhaps it would be better if IE only deferred sending the POST request body if the URL is the same as one that required authentication before. Meaning the flag that indicates authentication is required should be associated with the URL and not the HTTP client as a whole. Specifically, when an authenticated URL is encountered it is entered into a map. The map is checked with each request. Additionally you might also factor out entries in the map that share a common base path keeping the map or list small.

  2. Jay says:

    you need to disable the client-side optimization

  3. Indulis Ergis says:

    Thanks a lot your page came ap in google and it gave me a clue in our mistery. I was always thinking that if you have ananymous acces anabled then there is no way that gets are acepted and posts are rejected. KB25104 gave me an answer. We have an applcation that uses windows integrated authentication and we call non intagrated access anonymous application in I frames.

    So for the integrated the anonymous is anchecked but for non integearted aplication both where checked. That ledded up in iis that all gets for non integrated application got processed as anonymous login but all posts as integrated. Unchecking inegrated

    authentification for non integrated web application solwes the problem and posts are processed anonymously as well as gets working previously before.

    You can send me an e-mail at iergis@harriscomputer.com

  4. edt says:

    FINALLY!!! I been banging my head on this observed behaviour for weeks!!!!!!

    I did network traces and crawled thru all sorts of forums for answers with absolutely no luck.

    This explains it soooo clearly.

    Even microsoft support from India is stung on this…in fact after a 4hr conference call with them…they gave up and said the anonymous account is stuffed somehow…they’ve gone back to their brains trust and are still researching this hehehe

    Thank GOD i found this blog…definitely deserves a bookmark…just baffled that all my previous googling didn’t show this blog

  5. edt says:

    its me again…

    The thing about this problem in our environment is that it consistently appears in TEST, but "works" in PROD.

    In PROD, after an idling period (perhaps linked to the HTTP keep-alive timeout?), this behaviour rears its ugly head. Users reported that if they close their shut & restart the browser, things are dandy again. I tried to confirm this but it failed consistently for me, no matter how many times I restarted the browser. It was only after I rebooted my workstation (for other reasons) did I manage to POST again. Note that I tried this exercise only once, so I don’t rule out that I may have been "trippin". But it sounds like KB831167 even tho we havent installed the associated hotfixes.

    I "sensed" this behaviour was coming from the client/browser, but it threw me off that PROD has the same configuration, and yet it worked in PROD.

    Any ideas, directions (for further diagnostics), explanation you could give to further shed light on this behaviour would be greatly appreciated = )

  6. Mitchell Wu says:

    Your explanations are so clear and insigtful.  And the mystery is solved.  Thanks a lot.

    Two comments:

    1)  KB 251404’s suggestions to fix on client side is not proper for a web site.  We can go out and ask every user to change their registry and etc..

    2) This problem does not happen on Firfox and IE 7.

  7. David.Wang says:

    Mitchell –

    #1 I gave two possible solutions. You can choose which is appropriate for a given situation.

    #2 Perhaps FireFox and IE7 does NOT do the same optimization by default… remember, performance optimizations are not required by the HTTP specification.

    If I were writing a modern browser and can assume today’s bandwidth as a minimum, then I would make this optimization as optional, turned-off by default, but can be turned on with a registry key.

    Because then people get the mixed-authentication benefits by default (which many people want), few people would suffer the bandwidth consequences, and for the minority that require the old behavior, they are likely in organizations that can turn on the behavior with a registry key using Group Policy.

    This makes everyone happy at THIS point in time.

    Remember, Microsoft had the same conversation with IE a decade ago and decided on a different default behavior… because the technology and bandwidth assumptions were different at that time.

    //David

  8. Michiel Rademakers says:

    I have exactly the opposite problem. Setup is a company intranet on which webapps live. I have some popup windows POST’ing data and immediately afterwards closing themselves. Because the window is dead on the second part of the authentication, the valid POSTS’s are not sent anymore, or at least that is what I think happens. The HTTP logs show data consistent with this.

    I tried to do a setTimeout before closing the window, but IE does not honour the expected behaviour.

    Have you any suggestions how I can force IE to use Windows Integrated Security without Anonymous Access?

    Thanks,

    M.

  9. David.Wang says:

    Michiel – You just need to set the web application to only use Windows Integrated Security, and web browsers like IE will behave as expected.

    If you want the server to accept both Anonymous and Authenticated access at the same time and then control the client to do one or the other, that is the wrong approach architecturally and it is not supported by any public RFC.

    //David

  10. Sourav says:

    I am facing same problem.form is not posting..how to fix without editing registry