Measuring Browser Performance with the Windows Performance Tools


We’ve recently discussed the performance characteristics of the Flying Images and A GPU-Powered HTML5 Flickr Photo Viewer samples across different browsers. On the Internet Explorer team we use the Windows Performance Tools to measure browser runtime performance. The Windows Performance Tools are among the most accurate performance tools available which is why they’re commonly used across the industry. In this post I will give an introduction to using the Windows Performance Tools with Internet Explorer.

Using the Windows Performance Tools you can measure not only the overall elapsed time of operations, but also look at the time spent in individual browser and operating system components. In this post we’ll show you how to use the Windows Performance Tools to record and analyze the performance of different browsers. We’ll look at how to understand Windows events, measure CPU and GPU activity, identify working set patterns, and view network activity. We wouldn’t recommend the Windows Performance Tools for the average user; however these tools are a great resource for developers and this post will help you get started using the toolset to understand browser performance.

Background

Microsoft Windows Vista and Windows 7 have an extremely low overhead infrastructure built into the operating system called Event Tracing for Windows which collects performance and system data. Event Tracing for Windows, or ETW for short, enables the Windows operating system and applications to efficiently generate events at runtime. There are hundreds of events that can be captured during execution and later analyzed by tools.

The Windows Performance Tools, or WPT for short, allow you to capture, visualize and analyze ETW traces for your operating system. WPT is part of the Windows 7 SDK and can be downloaded from here. During the installation you can scope the installation to just the “Windows Performance Toolkit”:

installation options dialog

First Profile

Once you have the WPT tools installed, you’re ready to start analyzing browser performance.

Let’s started by measuring the performance of Internet Explorer 8 navigating to http://www.bing.com

  1. Launch Internet Explorer and navigate to http://www.microsoft.com.
  2. Start an elevated command prompt
  3. From that command prompt execute

    xperf -start mytrace -on PerfTrack

  4. Navigate IE to http://www.bing.com and wait for five seconds after the page appears to be visually done loading and the browser reaches a quiescent state.
  5. Stop the trace by executing

    xperf -stop mytrace -d mytrace.etl

  6. Now launch the Windows Performance Analyzer, part of the WPT toolkit, by executing

    xperfview mytrace.etl

The last command launches the Windows Performance Analyzer and opens the mytracve.etl file which contains the ETW events that were recorded during the trace.

Performance analyzer tool.

The Windows Performance Analyzer displays the captured events. In this example you captured events from several providers. Clicking on the ‘ProviderIds’ drop down reveals the 4 providers:

  • PerfTrackMetadata
  • WinSATAssessment
  • Microsoft-PerfTrack-IEFRAME
  • Microsoft-PerfTrack-MSHTML

ProviderIds Dialog

We will go into more detail about providers later in this post.

Notice the time scale at the bottom showing time in seconds. Next we will zoom into the cluster of events. There are two ways to zoom in: a) Ctrl + Scroll-wheel on the mouse, or b) Click + Drag to select a timespan followed by Right-Click on the selection and select ‘Zoom To Selection’.

zoomed perf analysis graph

Next, we will hover over an event to reveal its tooltip listing details about the event. For example, hovering over the first yellow dot shows its tooltip:

tooltip listing details about an event

The EventName indicates that this event is the Navigation – Start event, which is the beginning of the navigation to bing.com. 

The tooltip of the second yellow dot reveals it as the Navigation – Redirect event. The redirect occurs because we navigated to http://bing.com and the server responds with a redirect to http://www.bing.com. As you know, it is common for sites to redirect users to the primary URL of the home page.

The third yellow dot is the Navigation – Stop event, indicating the navigation to www.bing.com is finished.

Next, we will measure the duration of the redirected navigation to get a sense of the overall time taken.

Click + Drag from the second yellow dot to the third. While dragging, the tooltip shows the duration between the two points in time:

measured time period on a perf analysis graph

NOTE: The elapsed time between the start and stop events is not meant for comparison among different browsers. Different browsers can name and define events as they see fit. Comparing elapsed times makes sense for a single browser. Attempting to compare them across browsers (or even different versions of the same browser) doesn’t make sense. For example, the ETW events exposed by another browser might have similar names like Navigation Start and Navigation End but they measure very different aspects of the generic term “Navigation”.

However, the elapsed time is interesting for comparing results from multiple traces of the same scenario using the same browser. For example, it is interesting to compare the navigation duration of bing.com and another site. Since the pages are very different the elapsed time will be different. Later, we will look deeper into understanding where the time is spent.

So far, we have looked at the “timeline” view displaying all events captures during the trace. We can drill into events by zooming in and looking at the tooltips of each event. Windows Performance Analyzer has more views that we will look at later.

Providers

We previously mentioned Providers that are the source of the events captured by the Windows Performance Tool Kit. In the previous example, we told xperf to capture events from the “PerfTrack” provider by specifying “PerfTrack” on the command line

xperf -start mytrace -on PerfTrack

We can use Windows Performance Tool Kit to capture and analyze every aspect of the Windows operating system. Each Windows component exposes numerous ETW events so that overall Windows has thousands of events. In addition each application can expose many more, easily leading to information overload.

Depending on the scenario that we are profiling, we chose events from a set of providers so that we focus our attention to the relevant events.

We get a listing of providers, by executing the following command:

xperf -providers

The full list has numerous providers, including Kernel Flags and Kernel Groups at the very end of the list. The following are of interest when looking at Internet Explorer:

  • Microsoft-IE
  • Microsoft-IEFRAME
  • Microsoft-PerfTrack-IEFRAME
  • Microsoft-PerfTrack-MSHTML

As well as the Kernel Group:

  • Latency

We can use several user mode providers at once, for example:

xperf -start mytrace -on Microsoft-IE+Microsoft-IEFRAME+Microsoft-PerfTrack-IEFRAME+Microsoft-PerfTrack-MSHTML

To use user mode and kernel mode providers at the same time take a look at Example 2 half way down the page.

Tracing Scripts

When we analyze the performance of any component, we run performance trace many times over. To facilitate and make it easier to capture the same traces we use simple batch script. For the examples in this post we used trace.cmd:

@echo off
set session=mytrace
if not @%1@ == @@ set session=%1

xperf -start %session% -on PerfTrack
if not errorlevel 0 goto :eof

echo.
echo Performance Trace started.
echo.
echo When done with profile actions,

pause

echo.
xperf -stop %session% -d %session%.etl
if not errorlevel 0 goto :eof

echo.
start xperfview %session%.etl

The basic idea of the script is:

  • Launch the script from an elevated command prompt
  • Reproduce the scenario that we want to capture when the script is waiting for us
  • Hit any key in the command prompt to let the script finish the capture and display the result

The following is a slightly more complex script tracek.cmd that also captures kernel mode events:

@echo off
set session=mytrace
if not @%1@ == @@ set session=%1

xperf -on Latency -f %session%kernel.etl -start %session% -on Microsoft-IE+Microsoft-IEFRAME+Microsoft-PerfTrack-IEFRAME+Microsoft-PerfTrack-MSHTML -f %session%user.etl

if not errorlevel 0 goto :eof

echo.
echo Performance Trace started.
echo.
echo When done with profile actions,

pause

echo.
xperf -stop %session%
if not errorlevel 0 goto :eof
xperf -stop
if not errorlevel 0 goto :eof

xperf -merge %session%user.etl %session%kernel.etl %session%combined.etl
if not errorlevel 0 goto :eof

echo.
start xperfview %session%combined.etl

Machine Wide Perspective

When we take traces with the second “more complex” script Windows Performance Analyzer shows a holistic view of the entire system during the capture including CPU usage, CPU usage by process, Disk I/O and utilization, process lifetimes, and hard faults along with the Generic Events we looked at before. Each of the additional one can be just as important as CPU usage in determining performance.

Performance Analyzer graphs

We can select which graphs to display by clicking the black shape on the left edge which reveals the Frame List:

Frame list

We can also overlay multiple graphs to get a combined view. For example, Right+ Click on a graph and select another graph under “Overlay Graph”.

Summary Table

Now we will look at a very different view in Windows Performance Analyzer, the “Summary Table” view. Each graph has a “Summary Table” view that is tailored to data in the graph. For example, view the “Summary Table” view of the “Generic Events” graph via Right + Click on the “Generic Events” graph and select “Summary Table”:

right click menu

 Generic events summary table

We can drill down into the events by provider and reveal the individual event counts and time that they occurred.

Stack Walk

Stack walks provide another dimension when investigating performance profiles, since they let you drill down based on the “Weight”, meaning approximate CPU consumed. Since the profile data is based on sampling, it is always a statistical approximation of CPU consumed.

To enable stalk walks, we need to turn them on for the events we are interested in. In this case we are interested in the profile event which is part of the Latency group. For details on enabling stack walking on the event see Stack Walking in XPerf.

A Wide World to Explore

On the IE team we use the Windows Performance Toolkit along with internal tools daily to measure and understand the runtime performance of Internet Explorer. We welcome you to download the tools and start looking at browser performance using the Windows Performance Toolkit on your own systems.

Walter vonKoch

IE Performance Program Manager

Comments (23)

  1. Lost credibility says:

    You lost credibility as soon as your "navigate to bing" test states you need to start by going to "microsoft.com" – please stop pushing all the MSFT properties when totally unnecessary.

    I can ping bing.com without pinging any other site first.

    On an unrelated note, has .innerHTML been fixed in IE9 to work on all elements? (both setting and getting)?

  2. zvolkov says:

    Is there ETW provider that will show BHO-related events?

  3. @Lost credibility: They tell you to set it to some other site first so that the recored data is only for loading the page, not for loading the browser it's self. And why not choose microsoft.com, last I check this a MICROSOFT blog.

  4. Patrick Garies says:

    @Bill Gates II: While I don't see a use of microsoft.com as a big deal, example.com does exist for a reason. It's certainly less bandwidth-intensive.

  5. hAl says:

    Since Google has already chosen to branch the poor performing VP8 just a month to create a better performing codec and thus any implementation at the release of IE9 will likely not be compatible with future VP8 video will you reconsider supporting WebM with VP8 in IE9?

  6. Quppa says:

    Why do 'Windows Performance Analyzer' windows have fatter-than-normal (by 2px) border padding? I am genuinely curious. Cf. http://www.windows7taskforce.com/…/9

  7. Robert says:

    Posting about performance increases is pointless, IE is slow! and I mean really slow.

    When IE opens as quickly as Chrome, then post about performance increases.

  8. Matt says:

    Robert: IE loads as fast or faster than Chrome. Disable your slow browser add-ons and disable automatic proxy detection.

  9. Fiery Kitsune says:

    Please abridge blog posts with a "Read more" link on the main page. This is annoying.

  10. ken says:

    @Matt

    Still slower. Still less secure. Still less standards-compliant.

    Any other tips?

    @Bill Gates II

    Why should we need to "navigate" to microsoft.com? If the goal was to avoid the cost of loading the browser, why not simply load the browser. The first step should have been "Launch Internet Explorer", *NOT* "Launch Internet Explorer and navigate to http://www.microsoft.com, then navigate to http://msn.com/, and then navigate to http://msnbc.com/, and then navigate to http://windows.com/, and then navigate to http://live.com/…."

  11. Sam says:

    @Lost credibility, ken and patrick:

    You guys are missing the point of this post: Windows Performance Tools! If you are not going to post a sensible comment regarding the topic, keep your ugly and conceited opinions to yourself and stop leaving useless comments. IE8/IE9 (without pesky add-ons and toolbars) are as fast to load when compared to Chrome on Windows 7. There is NO PRACTICAL noticeable difference. I don't understand why it matters if Javascript execution is faster by approx 100ms in Chrome vs. IE. WOW! What a difference! Again, there is NO PRACTICAL noticeable difference. So, IE haters, stop spreading non-sense all over the internet about speed. What matters is that IE gives me Accelerators and other features which makes my life and experience as an end user much better.

    Further, no one should be irked whether this blog post mentions you to go to microsoft.com or google.com – the point is demonstration of Windows Performance Tools! Nothing less and nothing more. Are you guys really Microsoft technology based developers – because if you are – you guys need to re-evaluate what you are doing for a living (using technology that you hate for everyday living).

  12. AntiLuddite says:

    @Ken, IE8 on Windows 7/Vista is the most secure browser. It has Protected Mode support and SmartScreen Filter. Firefox is much more vulnerable than IE8. Or are you still using XP like a Luddite?

  13. Hand says:

    @Sam: There is a "pratical[ly] noticeable difference" for apps of the future. Microsoft had a reason for improving JavaScript execution in IE9.

    @AntiLuddite: It's a matter of definition and test arrangements whether you can call IE8 on Windows 7/Vista "the most secure browser". Its MIME-type handling and the XSS filter could be stricter, also there are some security technologies other browsers or extensions for other browsers have that are not found in IE8.

  14. hAl says:

    @Hand

    Extensions for other borwser are not part of those browser and allthough an extension like Noscript adds a big layer of security, 90% or more of Firefox users do not use it.

    A security layer like Smartscreen provided much broader security.  

  15. Robert says:

    @Matt

    I do not have any browser add-ons – But why should I have to disable loads of stuff to make IE work like it should to start with.

    It is still slower, less secure and less standards-compliant. Get over it.

    @Sam

    We post about these things to stop Microsoft getting lazy with IE development – There is still a long long way to go before they will have anything to be proud of.

    I would like to see IE regain some level of respect from the web developer community, but first Microsoft and the IE Team have to have some respect for the web developer community and the W3C.

  16. Jace says:

    Easy way to speed up IE and reduce its working set:

    1. Click Tools, Internet Options, Advanced tab

    2. Scroll down and uncheck the 'Enable third-party browser extensions' item

    3. Click OK

    Yes Henry, this will disable all the cutesy toolbars that hang like moss on your browser chrome…

  17. Neil Dunensach says:

    @Matt – no it doesn't.   A virgin-state IE 8 is slower than a virgin-state Chrome.  Especially right after a cold boot.

  18. hAl says:

    I guess no 7 week between preview this time around…

  19. Matt says:

    @Neil: What is so magical about your system that you think it represents all systems? IE is as fast or faster on all of mine.

    @Robert: Please take your rants and silliness elsewhere.

  20. Neil Dunensach says:

    @Matt: What is so magical about your system that you think it represents all systems? Chrome is faster on all of mine.

  21. Neil Dunensach says:

    @Matt: What is so magical about your system that you think it represents all systems? Chrome is faster on all of mine.

    Tested on various systems – XP (yes people still use it, shock horror) and Win 7.  Virtual and physical.  IBM and Dell hardware, plenty memory for the task.

  22. Max says:

    Konqueror is way faster than Chrome, duh!   Just kidding ^.^

    This is what I was looking for as tools like IE Dev tools, Firebug, YSlow, etc don't give you a picture of local resource performance, which will most likely be more relevant now with the Silverlight/Flex/RIA movement.  Chrome has Task Manager which is also good for this type of thing but doesn't show disk IO (not to say there isn't an extension for that already).  If you are developing apps, you have to test on IE, period.