The Internet Explorer Testcenter welcome page: Clarification & Corrections

Earlier this week, we published a number of new tests for web standards on our IE Test Center page, together with a harness to run them automatically in your browser as well as cross-browser pass rate statistics for these new tests. We quickly received web community feedback that the pass rate data in the first table was prone to misinterpretation.

What the Test Center Pass Rates Include:

The pass rates on the site are specific to only the 104 new test cases we just submitted to the W3C in conjunction with the first IE9 Platform Preview. As part of our regular involvement with the Working Groups across a number of W3C standards, we have identified a number of interoperability issues that are not yet covered by the standards body’s existing test suite.  We should note that the test results are only for the Windows versions of all the browsers in the test results table.

Pass Rates and Vendor-specific Feature Implementations

These tests were written and submitted as standard test cases, not IE9 Platform Preview demo pages.  Thus, the cases don’t include any vendor-specific prefixes. By contrast, our IE9 Test Drive samples include extra code to support the vendor-specific versions of certain features (e.g. -moz-border-radius for Mozilla’s implementation of CSS3’s border-radius).  Standards test cases do not and cannot include proprietary properties, methods, or markup.

In the case of border-radius, Opera 10.50 and recent Google Chrome builds support the un-prefixed version of the property. The latest major releases of Firefox and Safari do not. The pass rates for the current Firefox and Safari versions simply reflect that they require vendor specific prefixes and do not yet support the web standard markup. This wasn’t meant to imply that those browsers don’t have support for rounded corners using some markup but simply reflects the fact that they require -moz/-webkit CSS declarations to render them. From an interoperability standpoint, web authors cannot use the same markup across these browsers to achieve the same results today.

Test Case Fixes

Commenters on this blog and beyond also reported some bugs in a handful of the test cases we submitted for review on Tuesday.  Thank you!  This is exactly why there is a test case review process.  Feedback on the test cases helps the web development community.  We welcome it. We have updated the tests to reflect the initial feedback to ensure the cases are useful to the web standards community. The known issues and related pass/fail updates are described below:

  • CSS3 Selectors
    • The :nth-child-selector and CSS comments test case assumes that comments can occur between the multiplier and the n term i.e. that in the (an+b) expression, a and n are separate tokens. This was a bug in the test case. The IE9 Platform Preview should ignore this selector and thus fail the test case. Firefox 3.6, Opera 10.50, Safari 4.0, and Chrome 4.0 all pass.
  • DOM Level 2 Style
    • The test case verifies that @import rules inside an @media block are ignored but the case had a bug and checked for that result improperly. The first IE9 Platform Preview now fails this test.  Firefox 3.6 passes this test.  Opera 10.5 still fails the test. Safari 4.0 and Chrome 4.0 still pass this test.
    • Both the Syntax for backgroundRepeat and negative border width test cases had Javascript in them that mistakenly used an IE-only Javascript extension allowing elements to be referenced directly using their ID attribute value, which is supported for compatibility reasons.  The JavaScript in the test case was changed. The IE9 Platform Preview still passes both test cases.  Firefox 3.6 now passes both of these test cases.  Opera 10.50 still passes both tests.  Safari 4.0 still passes the backgroundRepeat test and still fails the negative border width test.  Chrome 4.0 still fails both test cases.

We thank the community for the feedback we’ve received so far. We have updated our tests, and the pass rate data, and submitted these updates to the standards body.

This is part of the standards process: a conversation leading to consensus. The one request we have is to please have these conversations within the W3C working groups instead of on various blogs.  We have a link to each working group’s mailing list in the table on this blog post.  We can have a healthy discussion about each test case in a threaded, archived format where everyone interested can participate and anyone can follow-along. 

What’s Next

As we continue developing and testing IE9, we will continue submitting our test cases to the W3C and the wider community. Conformance and interoperability can only be measured and achieved with the help of solid, complete test suites. Modern browsers should be able to demonstrate their conformance to web standards.  Test suites that identify cross-browser incompatibilities help all browsers make progress.

Jason Upton
Test Manager

Comments (33)

  1. Anonymous says:

    @ Jon Rimmer

    Stop bitching. really.

    this IS a platform *preview* with some "new" test cases that shows that IE team is not only aiming to reach where others are but also committed to improve the whole situation.

    ONCE IE9 is in beta stage, THEN you can bitch about all the test cases, acid3 and what not.

  2. DanielHendrycks says:

    "Test suites that identify cross-browser incompatibilities help all browsers make progress."

    Like Sputnik 🙂 Thank you, IE, for making the web better bye supporting these wonderful free standards. 😀

    Any response to Haavards other post?

  3. Frank B. says:

    @Daniel and @Haavards

    I can only guess what Microsoft will do here.  A friend of mine sells H.264 hardware to the TV industry, which is THE standard for video in Hollywood.  I imagine that it won’t be much different than what Apple has done.

  4. DanielHendrycks says:

    "A friend of mine sells H.264 hardware to the TV industry, which is THE standard for video in Hollywood.  I imagine that it won’t be much different than what Apple has done." That does not mean IE should not include both. Buy anyway, I’ll wait for Microsoft’s response for now.

    Night 🙂

  5. Jon Rimmer says:

    So why doesn’t the table include the results of the entire test suite, rather than just the ones that you specifically wrote test cases for!? What use is the current table to anybody? "Hey look! IE passes all the tests we wrote ourselves!" Any other browser could produce a similar table by selecting tests that it passed and others failed. The only reason for that page to exist is to give a misleading impression that IE is beating other browsers in standards compliance.

    I don’t know, it seems like the IE team is really making progress towards shipping a decent browser, but then someone like yourself has to go and spoil it by engaging in this kind of stupidity. Don’t you realise how badly this reflects on your team, your product, and you personally?

  6. Ms2ger says:

    Jon Rimmer: Actually, IE doesn’t pass all tests in the test center at this point. For DOM2Style, Firefox even gets a higher score than IE. I for one appreciate the IE team’s effort to submit these test cases and to swiftly fix the problems that were found.

  7. Amtiskaw says:


    Only because, as explained in the blog entry, there were problems in the test cases themselves that once fixed caused IE’s score to drop.

    I appreciate the effort in submitting test cases as well. Like I said, the IE team seems to be making excellent progress and doing a lot of things right, which is why the blatantly disingenuous test centre page is so disappointing.

  8. Jon Rimmer says:

    And that was me above, only logged in, obviously 🙂

  9. boen_robot says:

    Go IE team, go!

    Now THAT’s the way to handle feedback – fixing wherever applicable, explaining wherever misunderstood, asking standards bodies wherever unclear ^_^.

    Keep up the good ear *thumbs up*.

  10. hAl says:

    If that guy from Opera want MS to support Theora he might do well to offer MS a guaranteed waifer of liability for possible patent issues. You know, if anybody decided to sue MS for another billlion or so dollars.

    They are never going to take the risk otherwise to introduce support for a new format.

  11. Jason [MSFT] says:

    @John Rimmer

    We were simplying trying to make it easier for you guys by providing both a summary and the individual test case results.  I also got feedback on the IE Blog during IE8 that people really didn’t like clicking hundreds of times just to all get the same results.  It seemed more efficient for the IE Blog readers to provide both summary and individual results for these new cases we just wrote.  That’s all.

  12. Sylvain Galineau [MSFT] says:

    @Jon Rimmer, you are correct: we do indeed pass our own testcases. Because we write them in order to pass them !

    But the first public build of IE9 passing more testcases in this limited subset than any other individual browser is not as interesting at this early stage (I think) as the fact that cross-browser standard interop issues exist and remain relatively easy to find. ACID3 simply ran through dozens of such known issues, most of them with IE. But browser vendors will not be done when they all reach a score of 100 in that other limited test harness. There are many more such issues, across all browsers and across standards old and new.  

    When we find these issues, we will report them using testcases. And we do aim to pass all of the latter.

  13. Tim Snadden says:

    Any progress that IE makes is good. The disingenuous comparisons with other browsers are galling. For YEARS IE has been ruining the web and forcing everyone to dumb things down to its level. How about acknowledging that? How about a sincere apology? How about acknowledging the destructive effect of all of the attempts to subvert standards in the past? It’s like Darth Vader rocking up and wondering why everyone isn’t happy he came to the party. Have a little shame for the awful things you’ve done.

  14. Tim Snadden says:

    In the article you said "data in the first table was prone to misinterpretation."

    Take responsibility. The word you are looking for is ‘misrepresented’.

  15. Lars Gunther says:

    Just a quick note. If you publish something on your blog, you will get responses on other peoples blogs. That’s the nature of this beast we call the web.

    If you want feedback on mailing lists only, publish your stuff to mailing lists only.

    Anyway, good to see that you have listened and updated the test center page.

  16. lucideer says:


    You said: "If that guy from Opera want MS to support Theora he might do well to offer MS a guaranteed waifer of liability for possible patent issues."

    I’d highly advise you to look into the reasons behind campaigns for Theora codec support before making what is quite possible the most completely uninformed comment I have ever seen on the issue.

    Maybe my sense of humour is a bit off – was your post meant as a joke?

    Or are you actually seriously arguing AGAINST Theora because you think there may be patent issues?

  17. n4cer says:


    I’m pretty sure hAl was being serious since, given the evolution of the codec over the years, Theora could infringe on methods patented by MPEG-LA licensors or others and used in their respective codecs or MPEG codecs to which they have contributed. If Theora infringes and MS implements and distributes it, it probably won’t be long until MS is defending themselves in a court in east Texas.

    Concerning alternative codecs to H.264, I think VC-1 would be better than Theora. Sure, it isn’t free to implement, but it is comparable in quality to H.264, requires less CPU to decode, is supported in embedded chipsets, and doesn’t have per-stream royalties. The main downside is the lack of a standardized audio codec for it.

    In the future, maybe Dirac could fill this gap.

  18. ombarg says:

    @Tony Schreiner [MSFT] or @IE team:

    IE 9 preview currently implements 28% of SVG (*)

    Opera, Chrome, Safari and Firefox implement 94% , 87%, 82% and 72% respectively.

    What are your plans to the final IE 9 version? 30% 40% 95? Please can you share with us your goals with more precision?

    Thank you very much


  19. Jon Rimmer says:


    I find it hard to believe you had a deluge of requests asking for a summary of IE’s test case score, but only for that portion of the suite that Microsoft submitted themselves. Even if you did, making the information available is one thing, building what is essentially a marketing page for IE9 around it is quite another. What you should be doing is providing a summary table across the entirety of the test suite. If you also want to provide a table showing compatibility for new test cases you have submitted then you should provide as secondary information, with its context clearly marked and explained.

    @Sylvain Galineau

    Re. writing test cases to pass them, that was exactly my point! That is exactly why taking those test cases, putting them into a page and trumpeting the fact that you achieve a better score that other browsers is so dishonest. Particularly when many of those submitted tests have not yet actually been accepted into the test suites!

    I agree that development of test cases is important, and that they are key to achieving greater true cross-browser compatibility, as opposed to writing specs that vendors then go off an interpret in different ways. I fully applaud the IE team for developing these tests. What I don’t like is then taking these tests and using them to bash other browsers in a misleading way. If you think I’m being unreasonable, try reading through the test centre page, imagining that you are someone relatively unfamiliar with web browsers, the W3C, etc, just an average user or tech journalist trying to get a high level understanding, for example. Can you honestly say, hand on heart, that the page is intended to give you all facts in an transparent and unbiased way? Or is it an attempt to twist some figures so that, at first appearance, IE achieves a far higher level of standards conformance that it actually does? I’d be very interested to know.

  20. Sylvain Galineau [MSFT] says:

    @Jon Rimmer, I could ask you to imagine what less web-literate folks and ‘average tech journalists’ would make out of an arbitrary collection of 100 automated testcases each contributing the same 1 point to a 100 score that one particular browser happens to score lower at than the others. But  I doubt you or I will get anywhere by speculating on what others may or may not think. Arguing through hypothetical proxies is rarely productive.

    So I’ll just assume that this is your own interpretation. And it is an important one or we would not have bothered making this post. I can at least imagine, however, that your average tech journalist can find this blog, read Jason’s post as well as our comments. She can follow the links we posted to Haavard’s and Daniel Glazman’s commentary. That we are open and responsive to such unflattering criticism ought, I think, to factor in any judgment of our intent.

    I’m glad we agree that beyond message and perception issues, testcases are the way to go. Validating markup is important, but we all need the proper tools to validate browsers too. If ACID3’s popularity and the controversy begun by our test page tell us anything, it is that the need exists, it is pressing and it remains to be addressed.

    And the most important aspect of the solution will be trusting the results.

    Thank you for your feedback.

  21. Jeff Condie says:

    I am a novice trying to learn how to make good use of my time with this computer since I broke my back approx. a year ago. I was advised to start with google since it was more intuitive and easier to learn the basics on. I recently tried to switch to bing and have google my default browser. What a mistake, in one day I managed to get expelled because I forgot my password for hotmail’live’ which I don’t want but must have clicked something to get it. Trying to reset it is a maddening circular nonsensical  "excercise". I am a beginner But I was an active member of mensa until I had to start taking srong Painkillers for my back. What must someone do to get to where I can start complaining about what these accomplished computer users don’t like?  

  22. Jon Rimmer says:

    @Sylvain Galineau

    Well that’s a nicely political way of avoiding answering the question, I suppose. I imagine a proportion of users and journalists would use their understanding of high vs low numbers, red vs green, ‘pass’ vs ‘fail’, etc, to conclude that IE was passing more tests, and therefore more standards compliant, than its competitors. An impression given by a selective sampling bias in the data you are presenting. But hey, that’s just my crazy old hypothetical proxy’s opinion.

    As for my interpretation, I know enough about web development to have understood the figures presented straight off, but I still felt insulted at what seemed a deliberate attempt to mislead me. And much as I’d love to think journalists frequently trawl through developer blogs to scoop updates on last week news, I have rarely seen examples of such thoroughness.

    While it’s to your credit that you have recognised this criticism, the fact remains that the page in question is still up, apparently unaltered, still giving a potentially misleading impression of IE’s standards compliance vs other browsers. The IE team seems to often struggle between their better and worse instincts. Wouldn’t it be so much easier if, rather than things like this, then having to publish clarifications and debate with people like me, you simply didn’t put out this kind of ill-considered material in the first place?

    If you had instead published a test centre page that gave an overview of IE’s conformance across the whole test suite as well, as the new tests you wrote, then this whole mess might well have been avoided. I often feel sorry for the IE team due to the abuse you often receive on this blog and elsewhere, but sometimes you really don’t help yourselves.

  23. Matthew Raymond says:


    "If that guy from Opera want MS to support Theora he might do well to offer MS a guaranteed [waiver] of liability for possible patent issues. You know, if anybody decided to sue MS for another [billion] or so dollars."

    First of all, there are no known patent issues with Ogg Theora, and Opera doesn’t hold the existing, fully licensed VP3 patents, so for what reason would Opera offer protection against patent lawsuits, especially to a corporation that has more spare pocket change the entire net worth of their company?

    "Ah, but what about submarine patents?" you might ask. The problem with that reasoning is that there could be unknown patents on ANY technology Microsoft implements. Take SVG for example. Would you be surprised if there were patents on SVG that we don’t know about? Any new feature is just as much attack surface for patents as it is for viruses.

    Furthermore, do you honestly think that a random video patent is more likely to apply to Theora than H.264? If you look at the situation with ActiveX and Netscape Plug-Ins, dropping Netscape Plug-In support didn’t benefit Microsoft at all from the standpoint of patents. Any submarine patent would most likely need to have been filed prior to 1999, cover a technology unique to VP3/Theora, and the patent holder would have had to be sitting on the patent for more than a decade, in spite of it’s use in VP3 (and probably newer On2 codecs) and in WinAMP (which was owned by AOL).

    "They are never going to take the risk otherwise to introduce support for a new format."

    While I won’t argue that the risk is nonexistent, I doubt the risk of adding any other feature would involve any less risk. For instance, if the H.264 codec implementation has a security exploit that results in damage to customer data and property, they could be sued just as easily, and because H.264 is a newer codec, it has a longer period of exposure to submarine patent risk.

    The real reason for going with H.264 is that the format locks people into proprietary software if they want to be protected from patent lawsuits. Microsoft is essentially exploiting codec momentum and U.S. patent law to promote their own proprietary products.

  24. Amazed and disgusted says:

    Speaking of features that Microsoft has introduced …

    Just a few minutes ago, I found myself following a link, only to discover that it lead to a page that had since been taken down, and replaced with something belonging to one of the bad guys. A script is present on the new page that will not let one leave or close the window, until one "agrees" to a download.

    In the old days (last year), there was a simple, relatively easy solution when one ran into a page set up by the kind of lowlife who makes use of forced downloads. One would hit "alt-control-delete", and then end the application corresponding to the offending page. The good news for our good friends, the hackers, is that Microsoft is in their corner.

    Now, if one uses alt-control-delete to close one window, the system closes all of them. If one then goes back into Internet Explorer and hopes to open the other windows – happy surprise – one finds that this is an all or nothing deaal. One either gets back all of the pages, including the one with the forced downloads, or one gets back none of them, and accepts that a certain amount of one’s time and work will go to waste.

    Gee guys, thanks a heap.

  25. Stifu says:

    @Jon Rimmer: I fully agree with you.

    I’m a web dev and all, yet I was fooled by this page (the SVG test one) at first.

    Besides, I’d like to comment Sylvain’s point below:

    "I could ask you to imagine what less web-literate folks and ‘average tech journalists’ would make out of an arbitrary collection of 100 automated testcases (…)"

    Remember the Acid3? Most of the buzz comes from "less web-literate folks". They keep asking for more points and judge browsers based on their score, yet they have no idea what the test is really about.

    So yeah, I can totally see the impact of such a page on many non-dev people (that are simply browser enthusiasts or whatever), and I think that’s basically the whole point behind this page, ie: create buzz.

  26. Matt says:

    Stifu, Sylvain was originally making the same point: yes, ACID3 is misleading in exactly the same way as the chart of test results from the new tests.

  27. I can only guess what Microsoft will do here.  A friend of mine sells H.264 hardware to the TV industry, which is THE standard for video in Hollywood.

  28. @Jon Rimmer

    Not to validate or question the specific issues, but just to provide a data point – some of us journalists do indeed check out the blog and the comments to see the details, and having an update on just what’s changed is kind of useful 😉


    it’s not a numerical or % thing, but if you watch the SVG session from Mix by Patrick Dengler from IE and the head of the SG working group, they show a chart of what features are and will be supported now and for release, and they discuss what they’re not supporting and why (you can read some of the IE view on SVG here, though without the chart –

  29. Stifu says:

    @Matt: the main difference is that the Acid3 wasn’t made with the intent to mislead people (the Acid3 author admitted he regretted certain things, and learnt from his mistakes).

  30. hAl says:

    @Matthew Raymond

    [quote]The problem with that reasoning is that there could be unknown patents on ANY technology Microsoft implements[/quote]

    Actually on h.264 a lot of possible patents are known and listed in a patent pool in which a lot of the most important parties in video codec releated development are participating. Chances that patents on h.264 exist outside the reach of those patent pool companies that developed most of the existing video codecs and related software are slim.

    However chances are fairly high that Theora could inflict on several of those patents of the important video codec related developers.

    As there is no way to get protection for inflictionclaims on Theora it is unlikely that MS would consider using it unless someone would guarantee to a high degree that it would be save from patent claims.

  31. DT says:


    Personally, I have killed IE8 windows with Task Manager and had a single window close just as often as the entire thing closes.

    Either way however shooting IE in the head with Task Manager is not a supported method of closing the program and it’s ludicrous to rely on it functioning in a fully defined way. I suggest you turn your attention towards campaigning for changes to prevent you from losing control in the first place so that you don’t have to take such drastic steps.

  32. @ Jason Upton [MSFT]

    > Earlier this week, we published a number of new tests for web standards on our IE Test Center page,

    Those tests were presented as and labeled as "Professional-grade", you know.

    > This is exactly why there is a test case review process.

    So, those testcases should have been presented as, labeled as "unreviewed" and "not-yet-approved" tests.

    > The pass rates for the current Firefox and Safari versions simply reflect that they require vendor specific prefixes and do not yet support the web standard markup.

    I read your sentence many times and I can not reach/agree with your interpretation. The pass rates only reflect pass or failures for the batch of tests. Only the asterisk(*) note referred to vendor-specific prefixes for these tests.

    > These tests were written and submitted as standard test cases

    3 of them use invalid markup code:

    A few others use, rely on fractional pixels:

    border: 0.2in solid red;

    and margin-top: -2.4in;

    can create differences in rendering in perfectly-CSS2.1-compliant browsers. We already clearly explained and documented this very well.

    > (…) part of the standards process: a conversation leading to consensus.

    There has been a very wide consensus among web standards groups and W3C people that only valid markup code can create consistent, predictable page layout. I have never heard any solid commitments or clear statements from Microsoft IE Team on this. You and other IE Team colleagues often refer to "markup code" when it should be reworded as "valid markup code" in statements.

    Validity of markup code is not a purpose by itself, is not a panacea, is not a webpage decoration or a flavor-of-the-month feature: it’s a necessary minimum in webpages, in testcases and in any serious discussion about interoperability and web standards compliance for browsers.

    Gérard Talbot