Understanding Conditional Requests and Refresh

  • Article

Today's post is a collection of technical tidbits about conditional HTTP requests and the behavior of IE's Refresh button. It's probably of limited interest to most readers, but if you need to deeply understand either of these topics, hopefully you will find it helpful! 

Conditional Requests

Web browsers make two types of requests over HTTP and HTTPS—conditional requests and unconditional requests.

An unconditional request is made when the client browser does not have a cached copy of the resource available locally. In this case, the server is expected to return the resource with a HTTP/200 OK response. If the response’s headers permit it, the client may cache this response in order to reuse it later.

If the browser later needs a resource which is in the local cache, that resource’s headers are checked to determine if the cached copy is still fresh.  If the cached copy is fresh, then no network request is made and the client simply reuses the resource from the cache.

If the browser later needs a resource which is in the cache, but that response is expired (older than its max-age or past the Expires date), then the client will make a conditional request to the server to determine whether the previously cached response is still valid and should be reused. The conditional request contains an If-Modified-Since and/or If-None-Match header that indicates to the server what version of the content the browser already has in its cache. The server can indicate that the client’s copy is still fresh by returning HTTP/304 Not Modified headers with no body, or it can indicate that the client’s copy is stale by returning a HTTP/200 OK response with the new version of the resource.

Recently, someone asked why they see so many conditional requests in their server logs; they're setting proper far-future Expires headers and thus wouldn't expect IE to make any conditional requests for these resources.

There are a number of reasons why IE might make a conditional request for an item that is already in the cache:

  1. The cached item is no longer fresh according to Cache-Control or Expires
  2. The cached item was delivered with a VARY header
  3. The containing page was navigated to via META REFRESH
  4. JavaScript in the page called reload on the location object, passing TRUE for bReloadSource
  5. The request was for a cross-host HTTPS resource on browser startup
  6. The user refreshed the page


User-invoked Refresh

Now, in the case where the user refreshes a page, two of the multiple levels of refresh are relevant:

  • If the user clicks the refresh button or hits F5, IE will use OLECMDIDF_REFRESH_NO_CACHE.
  • If the user holds CTRL while clicking the button or while hitting F5, IE will use OLECMDIDF_REFRESH_COMPLETELY.

In the first case (Normal-Refresh), we will perform HTTP requests (conditional, if possible) to revalidate all of the resources on the page, regardless of freshness.

In the second case (Super-Refresh), we perform unconditional HTTP requests to redownload all of the content on the page, bypassing the cache altogether.

Notably, the latest versions of Firefox and Chrome both behave as IE does for both Normal Refresh and Super Refresh cases, so there’s a bit of a du jour standard for this behavior. 

There's actually a third type of refresh, which occurs when the user simply puts focus back in the address bar and hits ENTER, as if they were navigating to the page again. In that case, IE will use OLECMDIDF_REFRESH_RELOAD | OLECMDIDF_REFRESH_CLEARUSERINPUT. The first flag allows the browser to pull content from the cache if it's still fresh, while the latter flag clears any input fields on the document and resets the scroll position.

Refresh: Under the Covers

Internally, OLECMDIDF_REFRESH_NO_CACHE maps to the URLMon binding flags BINDF_RESYNCHRONIZE|BINDF_PRAGMA_NO_CACHE while OLECMDIDF_REFRESH_COMPLETELY maps to BINDF_GETNEWESTVERSION|BINDF_PRAGMA_NO_CACHE.  (OLECMDIDF_REFRESH_RELOAD doesn't set any URLMon flags.)  

These URLMon flags get turned into WinINET flags, and that actually influences the network behavior.

  1. BINDF_GETNEWESTVERSION gets turned into INTERNET_FLAG_RELOAD.
  2. BINDF_RESYNCHRONIZE gets turned into INTERNET_FLAG_RESYNCHRONIZE.
  3. BINDF_PRAGMA_NO_CACHE gets turned into INTERNET_FLAG_PRAGMA_NOCACHE.

These flags will influence WinINET’s behavior:

  • If INTERNET_FLAG_RESYNCHRONIZE is set:
    • WinINET will send an If-Modified-Since or If-None-Match request header to allow a 304 response.
    • WinINET may add a request header to help ensure that an intermediary (proxy) does not return a previously-cached result (see below)
  • If INTERNET_FLAG_PRAGMA_NOCACHE is set
    • If the request is going through a proxy or is HTTP/1.0, Pragma: no-cache is added. If the request is not going through a proxy and is HTTP/1.1, then Cache-Control: no-cache is added.
  • If INTERNET_FLAG_RELOAD is set:
    • WinINET will bypass the cache (redownloading all entries)
    • WinINET will not send an If-Modified-Since or If-None-Match request header on these requests (Unconditional request; server cannot return a HTTP/304).
    • WinINET will add a request headerto help ensure that an intermediary does not return a previously-cached result. 
      • If the request is going through a proxy or is HTTP/1.0, Pragma: no-cache is added. If the request is not going through a proxy and is HTTP/1.1, then Cache-Control: no-cache is added.

When the window.location.reload method is called, the different refresh flags are set depending on the value of the bReloadSource parameter:

  • True: OLECMDIDF_REFRESH_COMPLETELY|OLECMDIDF_REFRESH_CLEARUSERINPUT|OLECMDIDF_REFRESH_THROUGHSCRIPT
  • False: OLECMDIDF_REFRESH_NO_CACHE|OLECMDIDF_REFRESH_CLEARUSERINPUT|OLECMDIDF_REFRESH_THROUGHSCRIPT

 

Refresh: Resources that come down after page load

In IE9 and earlier, resources that are downloaded after the page downloads (e.g. XHR requests: https://www.stevesouders.com/blog/2009/08/11/f5-and-xhr-deep-dive/, or items pulled down by JavaScript) are not tagged with the BINDF_ flags that are set on resources that exist within the plain markup. This may be deemed undesirable: https://stackoverflow.com/questions/6775759/hard-refresh-and-xmlhttprequest-caching-in-internet-explorer-firefox/6879413 although the question of how long a "refresh" flag should persist is an open one.

Update: IE10 now will apply cache busting flags when XHR is used after a page is refreshed with CTRL+F5.

-Eric