Over the past year, the
structured feedback you provided while using the IE9 Platform Previews,
Release Candidate has been invaluable to helping us build the most reliable
version of IE ever. The error reports that you send us via
Windows Error Reporting and other methods help us fix the most impactful
issues experienced by users. Your real-world browsing enables us to identify reliability
problems in ways that cannot be matched by internal testing.
In this post, we take you through our process of engineering IE9 reliability.
We describe the various channels you’ve used to send us feedback. We describe
how we categorize and prioritize the data to address the most impactful issues first.
We conclude by sharing data of our own that show how IE9 reliability has improved
We encourage you to keep sending feedback through our telemetry systems as you use
the final IE9 release. Ongoing error reporting allows us to continue to deliver
reliability improvements to IE9 through
future Cumulative Updates.
Reporting the Crash through Windows Error Reporting
If you encounter
crashes in IE, features such as
Automatic Crash Recovery help mitigate the impact of these crashes,
but we want to eliminate their occurrences in the first place. We use several different
mechanisms to gather information about the crashes. One of these mechanisms is Windows
For users who opt-into
Windows Error Reporting, IE collects information about the state of the
browser when the crash occurs and packages the information into an error report.
This information helps developers debug the root cause and fix the crash. Internet
Explorer can also send error reports if you encounter hangs while browsing. We’ll
cover the topic of hangs in a future post.
Keep in mind that Microsoft
protects your privacy when submitting error reports. IE asks for your permission
before securely transmitting error reports back to Microsoft.
Windows Error Reporting dialog
Throughout IE9 Beta and RC, we used additional mechanisms such as the Send Feedback
Tool and Microsoft Connect to listen
to your feedback on IE’s reliability. These mechanisms may end up reporting duplicate
issues back to us but we internally de-duplicate reported issues. Your reports help
build a large and statistically significant volume of data that allows us to determine
the reliability issues that are meaningful to address.
Understanding How IE Crashes
Once we aggregate a full sample of crashes, we categorize the data based on the
source of the crashes. This helps us generally understand what areas are affecting
IE’s reliability more than others. We devote resources to driving improvements in
The following categories contributed to reliability problems during IE9 Beta:
IE Code Bugs
Faults in IE code contribute to a sizable portion of the overall crash volume. The
ratio of IE crashes to third party crashes is higher in pre-release versions (Beta
and Release Candidate) due to the amount of new code introduced.
While add-ons are a
core part of IE’s browsing experience, they are also the
primary cause of reliability problems in IE. Add-on related crashes
typically occur upon opening a new tab or upon IE startup. Some add-ons
are incompatible with newer versions of IE and may cause crashes.
Many of today’s Web sites use
ActiveX Controls like Flash, Silverlight, and QuickTime to display interactive
content and video. Since ActiveX controls are essentially Windows applications that
run in the browser, poorly-written controls can cause the browser tab to crash or
transition to hardware accelerated graphics is highly dependent on the quality
of the graphics card drivers in the ecosystem. We found that graphics card drivers
caused 41% of the crash reports we received from the Send Feedback Tool during IE9
Beta. These crashes manifest in similar scenarios as add-on crashes so they have
a very noticeable impact on regular browsing.
Other Third Party Software
A wide range of software programs, such as antivirus tools and custom download managers,
can affect IE’s reliability.
Drawing the “Failure Curve”
The next step is to organize the data from each category in such a way that we can
systematically investigate the crashes. We prioritize the issues using “failure
curves,” which are bar charts where each bar represents a unique crash and
the crashes are ordered from most frequently encountered to least.
After we construct the failure curve, we investigate each unique issue starting
with the most frequently occurring issues at the top of the curve. The issues in
the tail end of the curve are generally encountered in specific hardware and software
configurations. They may also be important to fix, but we start with the top issues
to ensure that we maximize the impact of our efforts.
The above failure curve example shows how we’ve successfully addressed the top 50
crashes in IE code from since IE9 Beta.
Ensuring Reliable Experiences
Through the work we do on the IE team and with third party developers, we
continue to improve the reliability of browsing experiences by addressing a majority of reported crashes. This section describes how we
make the final push for each category:
Internal Reliability Issues
We log bugs for the most frequent crashes in IE code from the Beta and Release Candidate.
We address them as we continue to monitor the volume of data for new issues.
We also combine the various sources of telemetry to good effect. We use your Send
Feedback reports to help us reproduce the issues that we investigate via Windows
Error Reporting. A successful recreation of the crash helps developers zero in on
the root cause and helps testers verify that the crash is fixed.
The failure curve chart from above paints an accurate picture of the progress we’ve
made fixing IE issues. In total, we’ve fixed the top 85% of crashes in IE
code reported since IE9 Beta, and we’ll continue to combat the long tail in cumulative
The improvements in quality and reliability in IE9 are substantial. The results of
Stress and Longhaul tests continuously running on the final IE9 release run 6-7 times longer than
Third Party Bugs
We work with vendors from around the world to address the top third
party reliability issues from the IE9 Beta and beyond. We use the
WinQual program to share the error reporting information with vendors. We
ensure that they only have access to error reports relating to their own products.
Faulty add-ons or graphics drivers are known to cause IE to repeatedly crash in
important usage scenarios. There are two built-in mechanisms in IE that help us
work around these problems:
- Add-ons: We use the
Upgrade Advisor to disable any known incompatible add-ons so that you can launch
IE successfully if you previously had one of these add-ons installed. When the Upgrade
Advisor blocks an add-on upon startup, IE notifies you of the action and gives you
an option to check for an update to the add-on.
- Graphics Drivers: IE9 includes a “software fallback list” of known incompatible
graphics drivers, compiled from error reports and from internal testing. If you
use one of these drivers, IE will use software rendering instead of GPU rendering
for accelerated graphics. Running in this mode prevents the drivers from crashing
So far, the error reports that you’ve submitted helped third party vendors address
some of the top crashes from IE9 Beta. We updated the software fallback list for
IE9 RC and streamlined the Upgrade Advisor experience (details to come in an upcoming
Your Feedback Matters Every Day
We followed the approach described in this post to address the top crashes across
IE and various third party components from IE9 Beta and IE9 RC, thanks to the error
reports and other types of feedback you contributed to us. The result is a high-quality
IE9 release as indicated by the record scores in Stress and Longhaul reliability.
Reliability engineering is a closed feedback loop. After the final release we will
use the feedback from an even larger group of users to continue improving reliability.
As the Web and the software ecosystem evolve, we’ll work to build an increasingly
reliable experience by shipping fixes to top problems through Cumulative Updates.
Thanks again for using IE9 Platform Previews, Beta, and Release Candidate and submitting
feedback in all its forms. Your contributions help make IE more reliable every day.
—Herman Ng, Program Manager, Internet Explorer