Changes in WinHttp on Windows 7 and onwards wrto HTTP/1.0

  • Article

I was working on an issue where an exe built based on https://msdn.microsoft.com/en-us/library/windows/desktop/aa383144(v=vs.85).aspx connects to a site hosted on Linux server. It uses NTLM authentication to authenticate against the squid proxy and gets 407 Proxy authentication failed method. The exe runs on XP and vista systems but gives this issue on Win 7 systems. The same site when access through browser IE/Firefox works fine.

My colleague Prashant Phadke, from the WinINet/WinHTTP escalation team helped find out the root cause. We collected Network and WinHTTP traces for this issue.
To collect WinHTTP traces on Windows 7, one can review https://blogs.msdn.com/b/jpsanders/archive/2009/05/28/how-to-enable-winhttp-tracing-on-vista-2008-and-windows-7.aspx 

We basically followed the below steps to get the data.

  • Open an elevated command prompt and type:

        netsh trace start scenario=internetclient capture=yes correlation=no report=no

  • Run the application and reproduce the issue
  • When the issue is reproduced, run the following command: netsh trace stop
  • When you stop the trace the location of the trace (.etl file) will be echo’ed to you.

The etl traces were parsed using Netmon.

After analyzing the traces, we found that the HTTP Protocol version was set to HTTP/1.0 in the call to WinHttpOpenRequest.

　　　　　 150297　　　　　 37.9147475　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.462 ::WinHttpGetIEProxyConfigForCurrentUser() returning TRUE

　　　　　 150298　　　　　 37.9309632　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::WinHttpSetOption(0x397a50, WINHTTP_OPTION_AUTOLOGON_POLICY (77), 0x21e3a8 [0x0], 4)

　　　　　 150299　　　　　 37.9309697　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::WinHttpSetOption() returning TRUE

　　　　　 150300　　　　　 37.9322592　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::WinHttpSetOption(0x397a50, WINHTTP_OPTION_PROXY (38), 0xfaa018 [0x3], 12)

　　　　　 150301　　　　　 37.9322701　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::WinHttpSetOption() returning TRUE

　　　　　 150302　　　　　 37.9337778　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::WinHttpConnect(0x397a50, "83.244.158.137", 80, 0x0)

　　　　　 150303　　　　　 37.9349361　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::Indicate Status 0x399218, 0x0, WINHTTP_CALLBACK_STATUS_HANDLE_CREATED (1024), 0x21e20c [0x399218], 4

　　　　　 150304　　　　　 37.9349403　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::WinHttpConnect() returning handle 0x399218

150306　　　　　 37.9361441　 2:34:43 AM 1/23/2012　　　　　　　　　 　　　　　 WINHTTP_MicrosoftWindowsWinHttp　　　　 WINHTTP_MicrosoftWindowsWinHttp:10:34:43.478 ::WinHttpOpenRequest(0x399218, "GET", "DummyHTMLPageName.html", "HTTP/1.0", "", 0x0, 0x00000000)

 

Note that we can also see this in a Netmon trace, however the WinHTTP ETW trace helps understand the API flow and how WinHTTP exactly sees the users request. The ETW trace also helps in analyzing HTTPS requests where one needs to view the raw HTTP request/response.

This however was just one part. We also found that the proxy was also responding with Http/1.0 as below.

ServerIP ClientIP HTTP HTTP/1.0 407 Proxy Authentication Required (text/html)

The basic point is that WinHTTP behavior in Windows 7 requires both the Client to send & proxy to respond with HTTP/1.1 which clearly wasn't happening in this case.

 

Now let's discuss as to why the browser works in this scenario. IE uses WinINet under the hood rather than WinHTTP. If we look at the network traces we see that IE sends HTTP/1.1, but the proxy replies with HTTP/1.0. IE still accepts this behavior, because in the internet scenario there are countless number of clients and servers which still use HTTP/1.0.

WinHTTP strictly requires HTTP/1.1 compliance for keeping the connection alive and HTTP Keep-Alives are not supported in HTTP/1.0 protocol. HTTP Keep-Alive feature was introduced in the HTTP/1.1 protocol as per RFC 2616. The server or the proxy which expects the keep-alive should also implement the protocol correctly. WinHTTP on Windows 7, Windows 2008 R2 are strict in terms of security wrto protocol compliance. The ideal solution is to change the server/proxy to use the right protocol and be RFC compliant.

Once again, I would like to thank Prashant Phadke for his help on this issue. 

Hope this helps someone.

Useful Links
=========
https://msdn.microsoft.com/en-us/library/windows/desktop/dd569142(v=vs.85).aspx
https://blogs.msdn.com/b/jpsanders/archive/2009/05/28/how-to-enable-winhttp-tracing-on-vista-2008-and-windows-7.aspx
https://msdn.microsoft.com/en-us/library/windows/desktop/aa383144(v=vs.85).aspx