HTTP.SYS rejection of request lines not terminated by CRLF


I recently got the following question about an HTTP.SYS behavior in Windows Server 2003.


Question:


Hi David,


I have a major problem switching to W2K3 and IIS6 currently.


Major handsets in the market (e.g. Nokia) send WRONG HTTP requests – nothing to do against it! Wrong means not seperated with “\r\n”.


I’ve seen your post: http://www.webservertalk.com/archive120-2004-2-132028.html


We are a leading provider for mobile multipayer games – and the only service based on .NET technology. Using IIS6 and W2K3 is important for us.


Is there any solution in the mean time?


greetings from Hamburg/Germany,


Answer:


I hope this illustrates the nasty problem when clients and/or servers do not follow public specifications like HTTP correctly – customers get caught in the middle and usually the server-side has to budge since there are millions more clients than servers to fix. However, realize that budging the server-side simply emboldens the clients to be even more broken in the future, destroying the underlying benefit of a public specification and endangering public communication protocols as we know it.


Now, the behavior being described is the default behavior for HTTP.SYS on W2K3. Due to customer pressure, HTTP.SYS added some features that allowed more relaxed HTTP parsing (i.e. hacks that allowed various broken clients to work). One of them includes allowing request lines to be terminated by only \r or \n instead of the spec-mandated \r\n.


None of these hacks are enabled by default. You must enable them separately yourself. I am inquiring the HTTP.SYS team about any KB articles similar to http://support.microsoft.com/?id=820129 which will describe these new settings and recommendations, and when I get an answer I will post it as follow-up. Stay tuned.


//David


FYI: You can also post questions like this to the microsoft.public.inetserver.iis newsgroup using a standalone news reader or a web news reader like http://www.microsoft.com/communities/default.mspx .

Comments (7)

  1. Mihailik says:

    Are you sure HTTP/1.0 doesn’t allow CR without LF? If my memory is not wrong, it does allow.

    If so, this feature must be set ON without any doubts — for backward compatibility with HTTP/1.0.

  2. David.Wang says:

    HTTP.SYS is correct in its default configuration. Both HTTP/1.0 and HTTP/1.1 specify that CRLF terminates the request line and request headers.

    See:

    1. HTTP/1.0 – RFC 1945 – http://www.w3.org/Protocols/rfc1945/rfc1945 – Search for "Page 25"

    2. HTTP/1.1 – RFC 2616 – http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1 – Search for "Request-Line"

    //David

  3. Mihailik says:

    Thank you very much, David. This information is exciting for me.

    I had built pure C# HttpListener component with CR/LF/CRLF variance in mind. As you show this was not really required feature.

  4. Mike says:

    " Media subtypes of the "text" type use CRLF as the text line break

    when in canonical form. However, HTTP allows the transport of text

    media with plain CR or LF alone representing a line break when used

    consistently within the Entity-Body. HTTP applications must accept

    CRLF, bare CR, and bare LF as being representative of a line break in

    text media received via HTTP."

    Does that apply to the headers or not? I’ve recently written some http code, and have discovered quite a few servers that use only n to seperate fields.

    It really is quite annoying when people go against the standards…

  5. David.Wang says:

    No, what you posted does NOT apply to headers. The RFC is very clear about what is allowed in the request headers versus the request entity body.

    Request line and headers must be delimited by CRLF (see my HTTP/1.0 and HTTP/1.1 links in the above comment).

    Request headers are separated from Request entity body via an additional CRLF. Thus, the first double CRLF encountered in an HTTP request demarcates request "headers" from the entity body.

    Request entity body can be anything you want, and this is what your paragraph refers to (media content has to refer to the entity-body, which is what is being transported — request headers are NOT what is being transported by HTTP; they are metadata that merely describe the entity body).

    Since request entity body is arbitrary, HTTP defines the Content-Type header to help clients figure out what type of data the server sent.

    It is saying that when you have Content-Type: text/* , then plain CR or LF must be allowed in the corresponding entity body. This means nothing to the request/response headers, though. Request/response line and headers must be CRLF terminated, and HTTP.SYS enforces this.

    //David

  6. David.Wang says:

    Ok, I got the URL link for the new HTTP.SYS settings introduced in W2K3SP1. http://www.microsoft.com/downloads/details.aspx?FamilyId=A3DA3D7F-18C7-45CE-A47A-ED747DACEF34&displaylang=en

  7. David Wang says:

    I recently sat down and thought a little about the typical user experience when troubleshooting IIS6,…