Note: The “brain dump” series is akin to what the support.microsoft.com team calls “Fast Publish” articles—namely, things that are published quickly, without the usual level of polish, triple-checking, etc. I expect that these posts will contain errors, but I also expect them to be mostly correct. I’m writing these up this way now because they’ve been in my “Important things to write about” queue for ~5 years. Alas, these topics are so broad and intricate that a proper treatment would take far more time than I have available at the moment.
When you navigate to a web address like www.bing.com/help in your browser, in most cases, the browser (or more specifically, WinINET, the network stack below IE) will immediately convert the hostname component of that URL (in this case, www.bing.com) into a numeric IP address like 220.127.116.11. It performs this magic by sending a query to a Domain Name System (DNS) server that then returns one or more address records for the target hostname. The browser then establishes a TCP/IP connection to the target IP and then uses HTTP (and/or SSL/TLS) to transfer web traffic over that connection.
Many people assume that a hostname will only return one target address, but in practice, it will often return several. For instance, on my PC, a DNS query for www.bing.com returns the following:
The first two addresses are IPv6 addresses, while the latter two are IPv4 addresses. Prior to Windows 2000, Windows would randomly select one of the addresses and attempt to use it; after Windows 2000, the first returned address is always returned first. The change was made to support the idea that the DNS resolver will attempt to order the address list by placing “preferred” addresses first. For instance, the list might be sorted based on the resolver’s knowledge of locality or connection throughput to a given server.
Windows Vista and later sort the IPv6 addresses to the front of the list by default (In the comments below, Lionel provides a netsh command to control the default).
While WinINET will attempt to connect to the first address, if that connection fails, that entry in the address list will be marked as “bad” and a connection will be attempted to the next returned address. This process repeats until a connection is made, the address list runs out of candidate addresses, or the retry count limit is reached (INTERNET_OPTION_CONNECT_RETRIES defaults to 5). If a connection is not made, the browser will show a “Page could not be displayed” error message.
The linear process of failover can lead to performance problems, if a DNS returns many records which must be tried before a working address is reached. Each connection attempt could take as long as 21 seconds. The most common case where this problem occurs is when a server returns IPv6 addresses to a client that cannot successfully make IPv6 connections due to network configuration problems. A proposal (oddly named “Happy Eyeballs”) was made to address such problems by more quickly attempting to contact IPv4 addresses if IPv6 addresses are slow to resolve. As far as I understand it, Firefox and Chrome utilize a variant of this notion. You can read about how Windows 8 improved IPv6 resiliency and performance over on the B8 blog.
Address List Caching
For performance reasons, WinINET caches address lists using an in-process memory cache; this cache allows reuse of recently used addresses without resolving them again. DNS resolutions are also cached by Windows' DNS Resolver itself, but retrieving them from the Windows cache requires an RPC call, which, while much faster than issuing a DNS request on the wire, still takes some time.
In Internet Explorer 9 and earlier, the per-process cache could hold up to 32 entries; in IE10 and later it holds up to 256 entries. The cache ignores any TTL (time-to-live) specified by the DNS server’s response. Instead each list is stored for 30 minutes by default; the DnsCacheTimeout registry key can be used to adjust this value. (WinHTTP, another Windows HTTP stack used primarily by services, has different defaults; it caches up to 32 records for up to 5 minutes).
It has been speculated that the WinINET DNS cache was created to attempt to prevent DNS Rebinding attacks, but this is a fallacy—the cache exists for performance reasons only, and is trivial to circumvent as a part of a rebinding attack.
What about proxies?
If the client computer is behind a traditional (“CERN-style”) proxy server, WinINET skips the DNS lookup for the target site and instead sends the request to the proxy server, allowing the proxy server to perform the DNS lookup on the client’s behalf. This is useful in some cases, for instance, the case where the client computer doesn’t itself have access to a DNS server that is willing to return records for non-Intranet sites. Of course, the browser may still need to use DNS even in this case—if your browser is configured to send traffic to CORPPROXY:8080, it will need to use DNS to look up the address of CORPPROXY. It’s possible that the DNS lookup for the proxy will return more than one address; if it does, the same pattern described above is used to decide which address to use.
However, in the unlikely event that you’re using a SOCKS-style proxy, WinINET must first perform the DNS lookup locally, sending that address information to the proxy server. That’s because IE only supports SOCKSv4, and the ability to rely upon the remote proxy to do DNS lookups wasn’t introduced until the SOCKSv4a protocol.