Strict HTTP Parsing

  • Article

In version 2.0 of the .Net Framework, we (System.Net) changed how we parse HTTP traffic.  The change has gone to a more strict parsing model.  One example, is we now require a CRLF (carriage return, line feed) at the end of each line.  While these changes were done to improve the security of our HTTP stack, it has the negative effect of causing some applications to see exceptions thrown where they didn't previously.   If you find the parsing of the HTTP headers to be too strict, you can revert the strictness of our parsing to improve app compatibility.  This is done via an application configuration file setting as shown below

Hierarchy:
<configuration> Element
<system.Net> Element (Network Settings)
<settings> Element (Network Settings)
<httpWebRequest> Element (Network Settings)

<httpWebRequest
maximumResponseHeadersLength="size"
maximumErrorResponseLength="size"
maximumUnauthorizedUploadLength="size"
useUnsafeHeaderParsing="true|false"
/>
 

Additional note about UnSafeHeaderParsing:

By default, the .NET Framework strictly enforces RFC 2616 for URI parsing. Some server responses may include control characters in prohibited fields, which will cause the System.Net.HttpWebRequest.GetResponse method to throw a WebException. If useUnsafeHeaderParsing is set to true, System.Net.HttpWebRequest.GetResponse will not throw in this case; however, your application will be vulnerable to several forms of URI parsing attacks. The best solution is to change the server so that the response does not include control characters.