Introduction to STRACE/HTTPREPLAY support tools


I'm Emmanuel Boersma and I'm Escalation Engineer on Internet Explorer for EMEA (I'm located in Paris). As a support engineer, I use and occasionally write troubleshooting tools. Today, I'm going to give some details on the following tools recently posted on Microsoft's download center:

Good and bad things about WININET logs

If you are dealing with HTTP issues in IE or if you write applications on top of WININET, you probably know WININET.DLL and WININET logging in order to generate HTTP traces. WININET logging is described in the following article : http://support.microsoft.com/kb/884931. The log produced contains details regarding calls to WININET API, data sent & received including clear text & encrypted data if you are using SSL.

Here's an abstract of a WININET.LOG for a connection to www.microsoft.com :

18:00:38.369 00000b60:<app> 001 InternetConnectA(0xcc0004, "www.microsoft.com", 80, "", "", INTERNET_SERVICE_HTTP (3), 0x00000000, 0x203668)
18:00:38.370 00000b60:<app> 001 HttpOpenRequestA(0xcc0008, "GET", "/", "", "", 0x137518, 0x00400000, 0x00203668)
18:00:38.372 00000b60:<app> 001 HttpSendRequestA(0xcc000c, "Accept-Language: en-us\r\nAccept-Encoding: gzip, deflate", -1, 0x0, 0)
18:00:38.679 000007a4:<---> 000 sending data:
18:00:38.679 000007a4:<---> 000 658 (0x292) bytes @ 0x217518
18:00:38.679 000007a4:<---> 000 00217518  47 45 54 20 68 74 74 70-3a 2f 2f 77 77 77 2e 6d   GET http://www.m
18:00:38.679 000007a4:<---> 000 00217528  69 63 72 6f 73 6f 66 74-2e 63 6f 6d 2f 20 48 54   icrosoft.com/ HT
18:00:38.679 000007a4:<---> 000 00217538  54 50 2f 31 2e 31 0d 0a-41 63 63 65 70 74 3a 20   TP/1.1..Accept:
18:00:38.679 000007a4:<---> 000 00217548  69 6d 61 67 65 2f 67 69-66 2c 20 69 6d 61 67 65   image/gif, image
18:00:38.679 000007a4:<---> 000 00217558  2f 78 2d 78 62 69 74 6d-61 70 2c 20 69 6d 61 67   /x-xbitmap, imag
18:00:38.679 000007a4:<---> 000 00217568  65 2f 6a 70 65 67 2c 20-69 6d 61 67 65 2f 70 6a   e/jpeg, image/pj
18:00:38.679 000007a4:<---> 000 00217578  70 65 67 2c 20 61 70 70-6c 69 63 61 74 69 6f 6e   peg, application
18:00:38.679 000007a4:<---> 000 00217588  2f 78 2d 73 68 6f 63 6b-77 61 76 65 2d 66 6c 61   /x-shockwave-fla
18:00:38.679 000007a4:<---> 000 00217598  73 68 2c 20 61 70 70 6c-69 63 61 74 69 6f 6e 2f   sh, application/
18:00:38.679 000007a4:<---> 000 002175a8  76 6e 64 2e 6d 73 2d 65-78 63 65 6c 2c 20 61 70   vnd.ms-excel, ap
18:00:38.875 000007a4:<---> 000 received data:
18:00:38.875 000007a4:<---> 000 1024 (0x400) bytes @ 0x217518
18:00:38.875 000007a4:<---> 000 00217518  48 54 54 50 2f 31 2e 31-20 32 30 30 20 4f 4b 0d   HTTP/1.1 200 OK.
18:00:38.875 000007a4:<---> 000 00217528  0a 50 72 6f 78 79 2d 43-6f 6e 6e 65 63 74 69 6f   .Proxy-Connectio
18:00:38.875 000007a4:<---> 000 00217538  6e 3a 20 4b 65 65 70 2d-41 6c 69 76 65 0d 0a 43   n: Keep-Alive..C
18:00:38.875 000007a4:<---> 000 00217548  6f 6e 6e 65 63 74 69 6f-6e 3a 20 4b 65 65 70 2d   onnection: Keep-
18:00:38.875 000007a4:<---> 000 00217558  41 6c 69 76 65 0d 0a 43-6f 6e 74 65 6e 74 2d 4c   Alive..Content-L
18:00:38.875 000007a4:<---> 000 00217568  65 6e 67 74 68 3a 20 33-32 31 30 31 0d 0a 56 69   ength: 32101..Vi

In addition to the data sent & received on connections, WININET log provides output regarding calls to WININET API which can be useful if you want to troubleshoot WININET application. The painful things regarding WININET are the following :

  • you need to get the appropriate debug build of WININET.DLL for your OS/configuration. You cannot use a debug build of WININET.DLL for XP SP2/IE7 and use it on Windows 2003 or Vista. Since WININET is also included in IE security fixes, it can be a real pain to find the appropriate debug version of WININET matching your configuration
  • the log produced may not contain any socket information indicating on which connection the data is sent/received
  • you cannot generate a WININET log for a process that has already been started without WININET logging enabled.

STRACE as an alternative to  WININET logging

The main goal of STRACE is to produce similar logging to WININET log without the above constraints. In addition STRACE is not limited to IE/HTTP. It can be used to trace any socket based application like, for example, Outlook Express. If the application uses SSL, STRACE will dump clear text data before encryption & after decryption. All you need to do is install the tool and double click on it : this will run a new Internet Explorer instance and generate a trace on the desktop for this IE instance. Here's the STRACE equivalent for above WININET log:

03/28/2007 13:33:48:758 - socket 0x000002cc created
03/28/2007 13:33:48:758 - connect socket 0x000002cc (65.53.196.57:80)
=====================================================
03/28/2007 13:33:49:305 - 719 byte(s) sent on socket 0x000002cc
=====================================================
      00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f   0123456789abcdef
0000: 47 45 54 20 68 74 74 70 3a 2f 2f 77 77 77 2e 6d   GET http://www.m
0010: 69 63 72 6f 73 6f 66 74 2e 63 6f 6d 2f 20 48 54   icrosoft.com/ HT
0020: 54 50 2f 31 2e 31 0d 0a 41 63 63 65 70 74 3a 20   TP/1.1..Accept:
...
=====================================================
03/28/2007 13:33:49:540 - 583 byte(s) received on socket 0x000002cc
=====================================================
      00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f   0123456789abcdef
0000: 48 54 54 50 2f 31 2e 31 20 33 30 32 20 46 6f 75   HTTP/1.1 302 Fou
0010: 6e 64 0d 0a 50 72 6f 78 79 2d 43 6f 6e 6e 65 63   nd..Proxy-Connec
...
=====================================================
03/28/2007 13:33:49:540 - 737 byte(s) sent on socket 0x000002cc
=====================================================

If you want to generate a LOG for a process other than IE7, Outlook Express for example, you'll just need to change the running process in STRACE.CMD by MSIMN.EXE  :

withdll /d:STRACE.dll_IE6 "%programfiles%\outlook express\msimn.exe"

The above command allows to trace SMTP/POP3/IMAP activity for Outlook Express on XP SP2.

Since STRACE is based on detours library (http://research.microsoft.com/sn/detours/), you can also "inject" the STRACE tracing DLL in an existing process using the "injlib" tool (see STRACE page for details).

HTTPREPLAY or how to parse and replay HTTP from a log file

Now, there is still one problem with either WININET or STRACE logging : you need to use your favorite editor and walk through the log …. Since a WININET or STRACE log can quickly take more than 100 MB, you may spend some time to find exactly what you are looking for.  This is where the HTTPREPLAY tool can help you. HTTPREPLAY provides the following services :

  • HTTPREPLAY parses STRACE or WININET log and generate a report
  • HTTPREPLAY can then be used to replay a live scenario based on the log

Once you have install the tool, you can browse to any STRACE and WININET log and open it using right-click -> Open With -> navigate to “c:\program files\httpreplay\httpreplay.cmd”. After the log is parsed, you’ll get a report similar to the following :

The HTTPREPLAY report provides :

  • summary of HTTP requests / responses
  • details for every request/reponse (click on GET request or HTTP status code to get details)
  • statistic allowing to measure number of request/response time, bytes sent & received…etc
  • logic to point common HTTP issues like : slow response from the server, failed request, bad content-length

Replaying HTTP from a log 

Now that the LOG has been loaded & parsed, HTTPREPLAY can be used to replay any response for the URLs listed in the log. The replay can be done in “proxy mode” or “direct mode”. “Proxy mode” just consists to set the replay tool as your proxy (by default : localhost:81). Direct mode requires that you edit the hosts file and enter all hosts referenced in the trace. I personally recommend to use proxy mode since this ensures every HTTP request will hit the tool. When you request specific content (by clicking on a URL in the report), you’ll see the requests/responses in the HTTPREPLAY command window :

17:53:14:623 #0 - GET http://www.microsoft.com/ [FOUND]
17:53:14:623 #0 - 127.0.0.1:2142 <- :81 (583 bytes / total : 583 bytes)
17:53:14:639 #0 - 127.0.0.1:2142 -> :81 (731 bytes / total : 1444 bytes)
17:53:14:639 #0 - GET http://www.microsoft.com/en/us/default.aspx [FOUND]
17:53:14:639 #0 - 127.0.0.1:2142 <- :81 (1024 bytes / total : 1607 bytes)
17:53:14:639 #0 - 127.0.0.1:2142 <- :81 (3356 bytes / total : 4963 bytes)
17:53:14:639 #0 - 127.0.0.1:2142 <- :81 (1460 bytes / total : 6423 bytes)
17:53:14:639 #0 - 127.0.0.1:2142 <- :81 (1536 bytes / total : 7959 bytes)
17:53:14:654 #0 - 127.0.0.1:2142 <- :81 (1460 bytes / total : 9419 bytes)
17:53:14:654 #0 - 127.0.0.1:2142 <- :81 (6072 bytes / total : 15491 bytes)
17:53:14:654 #0 - 127.0.0.1:2142 <- :81 (1460 bytes / total : 16951 bytes)
...

If the HTTPREPLAY command window doesn’t show anything then you are either not using the tool as your proxy or forgot to add a host in you hosts file.

Changing replayed content

Since we know how to generate a trace and replay it, wouldn’t it be interesting to change the way the content is “replayed” ? For example, what should I do to add an alert() in a js files ? Well, the 1st step consists to extract the responses in individual files. Running the following command will do the job :

C:\Program Files\HTTPREPLAY>httpreplay STRACE_IEXPLORE_PID_3932.LOG /xtract
...
18:10:44:393 50 files extracted (use /USEFILES switch to replay from files)

C:\Program Files\HTTPREPLAY>dir *.replay

Directory of C:\Program Files\HTTPREPLAY

31/05/2007  18:10           7 763 index.replay
31/05/2007  18:10              583 request0001.replay
31/05/2007  18:10          40 389 request0002.replay
...
31/05/2007  18:10              735 request0050.replay
 

The index.replay lists all URLs & response files :

C:\Program Files\HTTPREPLAY>type index.replay
http://www.microsoft.com/@request0001.replay
http://www.microsoft.com/en/us/default.aspx@request0002.replay
http://js.microsoft.com/shared/core/1/js/library.js@request0003.replay

You can “replay” from the response files using the following command: httpreplay /USEFILES. If you want to change the response’s content, you’ll just need to edit the required “*.replay” file.  Since HTTPREPLAY can use both STRACE log and index.replay, consider the following approach if you want to add an alert to a single JS file :

  • edit index.replay and only keep responses that you plan to change (example : http://js.microsoft.com/shared/core/1/js/library.js@request0003.replay)
  • edit the response file (request0003.replay), add alert(‘test’) in the 1st script block and save it (don’t forget to remove the content-length header as detailed in the tool’s documentation)
  • clear the IE cache and run the tool to replay from both log file and index : httpreplay strace_www-microsoft-com.log /USEFILES

The above approach can be useful for support engineers to reproduce problems, implement & test workarounds. It can also help web site developpers to write and test modifications offline.

I hope you’ll find the above tools useful and will get back to you with more blogs/info in the near future….

 - Emmanuel

SAMPLESTRACEREPORT.HTM


Skip to main content