Insight WPAD proxy settings on IE

  • Article

 

Backgrounds

The Web Proxy Auto-Discovery Protocol (WPAD) is a method used by clients to locate a URL of a configuration file using DHCP and/or DNS discovery methods. Once detection and download of the configuration file is complete it can be executed to determine the proxy for a specified URL. WPAD is documented in an INTERNET-DRAFT which expired in December 1999, although it is still supported by all major browsers. In this article, I will give you some deep understanding regarding how IE detects WPAD, and how to resolve some of the WPAD problems for IE.

The order in which IE detects wpad.dat file

Internet Explorer goes down a list of predefined detection methods:

1. DHCP (252 option)

2. DNS A record query/queries

3. NetBios

 

NOTE: Only applies when 'Automatically detect settings' in Internet Options -> connections -> Lan Connections is the only option checked.

 

Now, if DHCP is configured to provide the WPAD location, IE stops the detection and will make a GET request for the wpad.dat file and no further searching is done. This is true even if the DHCP 252 option is incorrect and a correct entry is configured as a DNS record. Please also be aware that IE still sends out the DNS query in this situation, even the DNS result won’t be adopted.

 

If the DHCP server does not provide the desired information, DNS is used. If, for example, the network name of the user’s computer is pc.department.branch.example.com, the browser will try the following URLs in turn until it finds a proxy configuration file within the domain of the client:

Notice: IE11 on Windows 7 does not try the DNS query.

If DNS is failed, a Netbios query for a machine named WPAD occurs.

 

Details of DHCP detection

When doing DHCP detection, IE tries to get the DHCP 252 option by calling DhcpRequestParams function. Based on MSDN article, DHCP clients store data obtained from a DHCP server in their local cache. If the DHCP client cache contains all data requested in the RecdParams array of a DhcpRequestParams function call, the client returns data from its cache. If requested data is not available in the client cache, the client processes the DhcpRequestParams function call by submitting a DHCP-INFORM message to the DHCP server.

So, DHCP-INFORM message is sent out only for the first time. The below is a sample of such a INFORM request/response.

55     10.0.0.101  255.255.255.255 DHCP          DHCP:Request, MsgType = INFORM, TransactionID = 0x33276787

56     10.0.0.1      10.0.0.101  DHCP          DHCP:Reply, MsgType = ACK, TransactionID = 0x33276787

  Frame: Number = 56, Captured Frame Length = 371, MediaType = ETHERNET

……

- Dhcp: Reply, MsgType = ACK, TransactionID = 0x33276787

    OpCode: Reply, 2(0x02)

    Hardwaretype: Ethernet

    HardwareAddressLength: 6 (0x6)

   ……

  + MessageType: ACK - Type 53

  + ServerIdentifier: 10.0.0.1 - Type 54

  + SubnetMask: 255.255.0.0 - Type 1

  + DHCPEOptionsVendorSpecificInformation:

  + DomainName: ctest.local - Type 15

  + DomainNameServer: 10.0.0.1 - Type 6

  + WPAD: https://consto.ctest.local/wpad.dat - Type 252

  + End:

 

For the next times, the 252 option is retrieved from DHCP local cache.

 

NOTE:

1.         IE won’t do IPV6 DHCP detection, only supports IPV4.

2.         IE tries to retrieve all Network interfaces’ DHCP settings.

 

Details of DNS detection

As same as DHCP detection, DNS also has its own cache. This also means IE needn’t sends out DNS query every time.

 

What else IE will do in WPAD detection?

After getting the URL of the wpad.dat file, IE will send a GET request to this URL to retrieve the wpad.dat file, this is a normal HTTP request just like you visiting a website from IE window. If the wpad.dat file is retrieved successfully, IE will cache the URL of the wpad.dat file into HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings. Also, the wpad.dat file will be cached in IE’s “Temporary Internet Files” folder if the HTTP response doesn’t contain any “Cache control” header. In most cases, such kinds of responses do not contain any “Cache control” header.

When IE starts the next time, it tries to retrieve the wpad.dat URL from registry (HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings), and then tries to visit this URL. Please just wait a minute, IE does find the cached wpad.dat file from “Temporary Internet Files”, so it won’t really send out the HTTP request anymore.

Why isn’t your wpad.dat file updated?

Based on the above WPAD detection mechanism, we always encounter the below two issues with wpad.dat.

l  We’ve updated the DHCP 252 option and pointed wpad.dat to another URL, or just changed the DNS entry and pointed to another web server, but IE never try to retrieve the wpad.dat file from the new URL.

As I mentioned before, IE always tries to retrieve the wpad.dat URL from registry (HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings). If it exists, IE doesn’t need to do DHCP detection or DNS detection. If it doesn’t exist, IE still needn’t to send DHCP or DNS request every time because of the DHCP cache or DNS cache, so that IE may still get the old wpad.dat URL.

l  We’ve modified the content of the wpad.dat file on the server, but IE still use the old content of the wpad.dat, even it doesn’t try to download the new wpad.dat file.

This is because the local cache in “Temporary Internet Files” folder. If the wpad.dat URL isn’t changed, IE won’t try to download wpad.dat file again if it is already cached in “Temporary Internet Files” folder, this is because of IE’s cache mechanism.

This also means IE won’t be aware of the modification of the wpad.dat file on your server.

How to resolve the problems?

If you are encountering the 1st problem, the only way is to clear all caches as below.

Clear DHCP cache

ipconfig /release

ipconfig /renew

 

Clear DNS cache

ipconfig /flushdns

 

Delete proxy cache from registry

HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings

HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\SavedLegacySettings

 

Delete IE cache

RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 8

 

If you are encountering the 2nd problem, we have two options as below.

l  On the web server which hosts your wpad.dat file, add “Cache-Control” header for the wpad.dat response and set to “no-cache”. For example, the HTTP header could be like below.

Cache-Control: no-cache

For more details regarding Cache-Control in HTTP protocol, please refer to https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

l  Clear IE’s cache folder(“Temporary Internet Files”) before starting IE

NOTE: To consider the browsing performance, the second option is preferred.

 

References

WPAD detection in Internet Explorer

https://blogs.msdn.com/b/askie/archive/2008/12/18/wpad-detection-in-internet-explorer.aspx

 

Web Proxy Autodiscovery Protocol

https://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol

 

Regards,

Zhixing Lv from APGC DSI Team