Using Meddler to Simulate Web Traffic

  • Article

As mentioned back in July, IE8’s new lookahead downloader has a number of bugs which cause it to issue incorrect speculative download requests.

The “BASE Bug” caused the speculative downloader to only respect the <BASE> element for the first speculatively downloaded script file. Subsequent relative SCRIPT SRCs would be combined without respecting the specified BASE, which resulted in spurious requests being sent to the server. Eventually, the main parser would catch up and request the proper URLs, but the spurious requests waste bandwidth and could cause problems for some servers.

When first investigating the speculative downloader problems, I decided to use the Meddler HTTP Traffic Generation tool to build some test cases. Meddler is a simple little tool that allows you to write JavaScript.NET scripts to emulate a web server. Meddler allows for precisely timed delivery of responses, and includes classes to enable basic fuzzing scenarios. The best part of Meddler is that you can use a single MeddlerScript (.ms) file to contain an entire test case, even if that test case is made up of multiple pages, images, scripts, and other resources. These .ms files can be shared with others, run across multiple operating systems, and attached to bugs or test harnesses for future regression testing. The test machine only requires the .NET Framework and Meddler installed, and does not need IIS, Apache, Perl, ASP.NET, etc.

Because the base issue was so simple, I was able to quickly build a simple MeddlerScript which demonstrates the BASE Bug. If you’d like, you can follow along using my MeddlerScript: PreParserBaseBug.ms.

The test script generates the following sample HTML:

<html><head><base href="https://ipv4.fiddler:8088/pass/"></base>
<script type="text/javascript" src="inc/1.js"></script>
<script type="text/javascript" src="inc/2.js"></script>
<script type="text/javascript" src="inc/3.js"></script>
<script type="text/javascript" src="inc/4.js"></script>
<script type="text/javascript" src="inc/5.js"></script>
<script type="text/javascript" src="inc/6.js"></script>
<script type="text/javascript" src="inc/7.js"></script>
<script type="text/javascript" src="inc/8.js"></script>
<script type="text/javascript" src="inc/9.js"></script>
</head>
<body> Test page.</body></html>

Note that I plan to watch the network traffic with Fiddler, and because traffic sent to localhost isn’t proxied, I will use “ipv4.fiddler” as an alias to 127.0.0.1.

When visiting the Meddler test page, the traffic from IE is as follows:

Screenshot of original incorrect network traffic

As you can see, there are spurious download requests containing the wrong path; these are shown in red as the MeddlerScript is designed to return failure for such requests. Later, the correct URLs are downloaded as the main parser encounters the script tags and correctly combines the URLs.

Today's IE8 Cumulative Update (KB974455) fixes the BASE Bug. After installing the update, loading the sample HTML results in no spurious requests-- each script URL is correctly relative to the specified BASE.

Screenshot of corrected network traffic

Please note that while the BASE bug is fixed, the “4k Bug” is not fixed by this update. If you want to view that bug in action, try this script: PreParser4kBug.ms. As it is a timing issue, you may need to reload the “hammer” page a few times to encounter the problem.

While Meddler is rather simplistic, it can be very useful for sharing test cases and simulating the behavior of web servers. You can use Meddler to build reduced test cases that reliably generate problematic HTTP responses.

Until next time,

-Eric