A GPU-Powered HTML5 Flickr Photo Viewer


With Internet Explorer 9, developers can build new classes of HTML5 applications that were previously not possible. We’re having fun building sample web applications that provide a glimpse into the types of experiences that hardware acceleration provides. In this post, we take a closer look at Flickr Explorer, one of the samples that we released alongside IE9 Platform Preview #2.

Flickr Explorer is written using standard HTML, CSS and JavaScript. It uses AJAX to asynchronously search Flickr based on your search terms and display image results through an interactive and visually compelling experience. You can zoom in, pan around, and open each photo’s Flickr page with by control+clicking.

With Internet Explorer 9, Flickr Explorer is generally able to maintain a near real-time responsiveness of 52fps (52 frames per second). In contrast, other browsers struggle to maintain 4-8fps, which is barely 15% the performance that Internet Explorer 9 provides in this particular scenario.

Flickr explorer demo

As we discussed last month in a post on the Flying Images demo, we redesigned the core of Internet Explorer 9 to be fully hardware accelerated. Internet Explorer 9’s display rendering subsystem uses the GPU for all graphics and text on web pages. Internet Explorer 9 moves graphics work that has traditionally occurred on the general purpose CPU to faster, more specialized hardware. Internet Explorer 9’s JavaScript engine takes advantage of multiple-processor cores to background compile JavaScript into machine code. Internet Explorer 9 uses modern processor instruction sets across the entire product. Taken together, these changes enable computations to occur faster and in parallel, freeing the CPU to spend time performing other operations.

Let’s take a closer look at the CPU and GPU activity graphs collected while running the Flickr Explorer sample to see more hardware acceleration in action. These traces were captured while zooming in on a selected Flickr photo, and use the same machine and methodology that we previously discussed.

First, results from Internet Explorer 8 are shown below. Like we saw with the earlier Flying Images demo, Internet Explorer 8 utilizes an entire CPU core (50% of this dual core machine) to animate the images on screen. Internet Explorer 8 does not utilize the GPU in this scenario.

IE8 CPU at 100% for the entire time period, GPU at 0 for the entire time period, 1 draw to the screen over the time period

Internet Explorer 8 can barely make one screen update every 0.253 seconds, which results in roughly 3.9 frames per second (FPS). This is clearly a very poor experience for the user.

Next, let’s check out how the new Chrome 5 beta handles this task. As you can see below, it doesn’t fare much better than Internet Explorer 8. Chrome is able to update the screen every 0.222 seconds, achieving a frame rate of about 4.5 FPS. Again, this results in a very choppy and undesirable experience. Chrome also does not utilize the GPU in this scenario.

Chome 5 Beta CPU at 100% for full time period, GPU at 0 for full time period, 1 screen update

Safari 4 handles the demo similarly to Chrome, earning a slightly lower frame rate of 4.2 FPS. Again with Safari, the GPU remains unused while the CPU remains a bottleneck.

Safari 4.0.5 CPU at 100% for full time period, GPU at 0 for full time period, 1 screen update

Next, let’s take a look at Firefox. We used the most recently Firefox nightly build, Minefield 3.7a5, for this analysis. Like each other browser tested, we ran Minefield in its default configuration (which means hardware rendering with D2D was not enabled). We will post comparisons with Firefox’s hardware acceleration when it’s on by default in their beta.

Firefox handles the demo much better than Chrome and Safari. The screen gets updated roughly once every 0.12 seconds, which results in a frame rate of about 8.3 FPS. While this is nearly double the score of Chrome, it is still quite slow and barely usable.

Firefox 3.7a5pre CPU at 100% for full time period, GPU at 0 for full time period, 2 screen updates

Finally, the results from Internet Explorer 9 Platform Preview 2 are shown below.  We can see that, unlike with the other browsers, the CPU handles this task with ease and has periods of frequent rest where Internet Explorer and your applications can performance additional operations, while the GPU renders Flickr Explorer to the screen.

IE9 Platform Preview CPU revs to 100 and drops to 0 several times, GPU revs to 100 and drops to 0 several times, 13 screen updates during time period

Internet Explorer 9 is able to update the screen once every 0.019 seconds on this mid-range configuration, which equals a frame rate of about 52 FPS.

When you play with the demo, the difference between Internet Explorer 9 and other browsers is crystal clear. While you can easily zoom in, out, and pan through the Flickr photos with Internet Explorer 9, it’s difficult to do this with the other browsers.

We’d like to extend a special thanks to the Flickr team for building such a simple, easy to consume services API. We were able to put this sample together in a couple of days using the same APIs that web developers program against today. The only difference is that we have higher expectations for the types of application that you can build through standard markup and these APIs.

We’re excited to see what new experiences web developers will create with IE9’s hardware acceleration!

Seth McLaughlin
Program Manager for IE Performance

Comments (69)

  1. Buddy Lindsey says:

    I am getting excited to start seeing GPU powered browsers. Good Job. Just might get me to switch back, maybe.

  2. Kieron says:

    Good Job.

  3. Agamemnon says:

    I understand why you didn't include the Opera results, since it even without any hw acceleration activated it is equally fast as IE9 PP #2 (50+ fps with version 10.53 on Windows 7 with a modest tripple core processor).

    My Firefox 3.6.3 did around 16 fps, while the non-beta Chrome 5 wasn't able to run it (no pictures was shown).

  4. Wurst says:

    Browser tests are interesting, because small bottlenecks can make a huge difference. They can show us with which problem people creating rich(er) web applications will be confronted in the future.

    What I would be interested in, is to know why IE8 and WebKit-based browsers like Safari and Chrome perform so horrendous at the Browser Flip test while current versions of Opera and Firefox fare quite well.

    On the other hand there is this "Chrome experiment" at mrdoob.com/…/depth_of_field which runs quite nicely on Chrome and Opera while not so much under Firefox (sadly, it won't run under IE9 properly even when spoofing the user agent [I tested it with the first preview version]).

  5. Randall says:

    Nice, and I certainly applaud the IE team's dedication to performance and making DOM-based animation "just work" without special work by authors or new APIs.

    It's also true, however, that you *can* do this stuff efficiently with HTML and CSS on Safari, with CSS transforms and transitions.  http://www.satine.org/…/snowstack.html is an example; it even has a fun 3D perspective effect on my computer.

    Does all this perhaps mean we'll see http://www.w3.org/…/css3-transitions or even http://www.w3.org/…/css3-3d-transforms in a future IE?  I'm sure the IE team can only do a certain number of revolutionary changes to IE at once, but I'll be a happy man when/if I see a Microsoftie join as an editor of one of those documents. πŸ™‚

  6. Jeffrey Gilbert says:

    ship this puppy.

  7. Mack D. Male says:

    I think its incredibly lame that you continue to exclude Opera from these posts.

  8. oh says:

    C'mon, where is Opera? Chrome and Safari have the same rendering engine anyway.

  9. Boring says:

    Until you compare HA browsers with other HA browsers these comparisons remain very uninteresting.

  10. Badger says:

    @Boring It's interesting that other browsers don't have HA enabled by *default*. Might indicate it's buggy in those browsers. Most end users will run the browser with its default config. So it's fairest to test them in their default config.

  11. Dom says:

    And Opera is obviously out, because the test runs at 20fps on my netbook…

  12. Dave says:

    Or Opera is not included because it has <3% of the market share.  Although, limiting comparisons to the top 3 competitors has the added bonus of making IE9 looking even more impressive.

  13. Lari says:

    Why did't you turn Direct2D on for Firefox 3.7a? It's just two clicks. Firefox should be faster than IE9 then. Oh, maybe that was the reason, never mind.

  14. F1LT3R says:

    Here comes the web-top!

  15. Phil says:

    And yet, small fonts look horrible in IE9 by comparison with IE8/Cleartype/GDI, and there are serious problems with highlighted text selection. So before you start crowing, please make the result look equivalently good.

  16. Johnnyq3 says:

    @Lari

    The test is meant to test default settings on a browser.  GPU acceleration is default on IE9, it isn't true with FF3.7

  17. Blah says:

    Yeah, it only takes two clicks in Firefox to get a browser that looks ugly, renders poorly, and crashes. No thanks… I'll wait for a stable release.

  18. fowl says:

    So how long until we get hardware acceleration in Silverlight?

    ok ok, I'll stop πŸ˜› I know it's a completely different team.

    Great work! I've got one question though, what sort/level of GPU are you testing? What are you expecting to work with? My graphics card, Geforce 6800GS gets a 5.9 on the WEI scale, but it's only dx9, although it has a WDDM driver… so.. would that be expected to work, work well?

  19. fowl says:

    Oh and I'm assuming that we'll have to structure our webapps in a certain way to get most out of IE9. Will MSDN (or something better) start including performance information?

  20. doesn't matter says:

    Where is Opera? FPS is over 50 all the time WITHOUT hardware acceleration.

  21. blah says:

    Microsoft can't mention Opera. They were shameless enough to name their JS engine Chakra. Gee, wonder where they got THAT from.

  22. EricLaw [MSFT] says:

    @fowl: Generally speaking, the goal is "same markup" so web developers will have a great experience without any particular "structure" for IE.

    In terms of the graphics card, 5.9 is a very good score compared to what we test with. My main machine (Lenovo x200) gets a 3.4 for "Gaming Graphics" and 4.1 for Aero.

  23. Infinte says:

    So this means DX was used EVERYWHERE?

    So please support direct manipulation on TEXTURE BITMAP DATA and USER-DEFINED SHADERS.

    Shaders can be provided in a seperate file and compiled by IE.

    for example:

    <link rel="-ms-shader" name="SHADER1" href="file.effect" />

    Then,in CSS:

    element {

     shader : "SHADER1(),SHADER2(5)"

    }

    scripting:

    if(element.msSh

    aders){

       element.msShaders[0].args[0] = 3

    }

    Texture manipulation:

    var tx = new Texture();

    var tx2 = imgElement.msToTexture();

    tx.setPenStyle(0xff0000,1,0); // red, width:1, solid

    tx.renderElements(Texture.POLYLINE, [0,0,0,1,1,1,1,0])

    element.setTexture(tx)

    It may be cool!

  24. Infinte says:

    @EricLaw: So you must implement a method for developers to draw images — i mean bitmaps — on IE.

    In other browsers, it's called <canvas>

  25. William J. Edney says:

    Personally, I think it's great that Microsoft is so focused on browser performance now, after so many years of being dead last on all performance benchmarks. Yes, it is a bit disingenuous to not turn on D2D in the latest Firefox builds, etc. but you know that's ok. The IE9 team deserves a hell of a lot of credit for getting this browser to rock as hard as it is doing, so I'm lovin' it. Remember – browser speed competition amongst vendors helps *all* of us web developers.

    So, now that you guys have all of this cool graphics power at your fingertips, maybe someone at Microsoft can chime in on IE9 supporting the following new CSS3 proposed standards:

    CSS 2D transforms

    CSS 3D transforms

    CSS gradients

    CSS transitions

    CSS animations

    CSS drop shadow

    All of these are already in Webkit (Safari/Chrome) – FF 3.6.X is currently only missing 3D transforms, transitions and animations and FF 4.X will definitely get transitions support.

    How about showing off even more by supporting these? πŸ™‚

    Cheers,

    – Bill

  26. Rob says:

    Lots of dazzle to divert us from the main show, Microsoft? Trying to make us forget all the things IE9 won't do? It's still over a year away and it won't handle properties all the others do today. This thing is dead. Give it up.

  27. Anonymouz777 says:

    We need "Web Worker" also for the more fast performance. It is possible to use the multi-core cpu and multi threads.

  28. Constantine says:

    But where is Opera in the comparison charts ?

    its also GPU-accelerated,

    its gives 60 FPS easily (of course),

    its not in beta (like FF3.7 which is on the test),

    and it has 40% market share in Russia / ex-USSR countries (while IE has there less than 20%, all versions combined).

  29. Hello says:

    @Constantine

    Opera is not using GPU/hardware acceleration yet. Just thought you should know. It's pure software rendering πŸ™‚

  30. webster58 says:

    weird .. Opera 10.53 60fps πŸ˜€ when mouse gone rampage i got 42 fps πŸ˜€

  31. Kevin Buchan says:

    OK, cool.  I am ready to come back to IE.

    Now, will I get it in Q3?  Q4?  Do I have to wait until 2011?  Chrome won't take that long.

  32. 6205 says:

    I want Slovak IE9 version in Windows Update this year!!!

  33. Brian LePore says:

    Maybe I'm mistaken, but I swore Mozilla said that there would not be a 3.7 and that many of the changes were being rolled in to Firefox 3.6.4. Hardware acceleration, while disabled currently in the nightlies, will be enabled for the release of 4.0. If you are going to test for the defaults of upcoming browsers why not test for the planned defaults and not just the current defaults?

    BTW, the new blog looks spiffy. πŸ™‚

    @Bill: CSS Gradients, while I would LOVE them, are not accurately defined presently. Mozilla's implementation is completely different from the Webkit version. Mozilla's is much simpler to work with. I don't recall off the top of my head, but if IE9 doesn't have background-sizing then I don't even want CSS gradients.

  34. avokicchi says:

    I turned on hw-acceleration in the latest firefox build and ran the flying images demo. Ran very, very choppy at first, seeming to freeze up randomly, and after a while stabilized, and became lightning fast. Oh and it screwed up the UI. So yeah, buggy as hell, but fast, not ready for testing, so I agree with the IE ream not testing this.

    Also, a post on specifically what parts of html5 / css3 will be supported would be nice, as the IE team has been especially vague about certain parts. It would stop this flurry of "will feature x be supported? " , "you MUST support feature y!!". I'm all for finalizing and improving the spec for some features, but that doesn't say much about whether IE9 will implement it. I'm pretty interested if websockets, canvas, box-shadow, text-shadow, css gradients, multiple backgrounds(or just layering in general) and the like will be supported in the final IE release.

  35. William J. Edney says:

    @Brian – yea, I know ;-). I was hoping that if Microsoft weighed in with "well, we're gonna implement gradients in IE9", that would give a kick in the pants to the working group to settle on a final syntax. The other working drafts that I mention do seem to be pretty stable between Webkit & Gecko, so I'd hope that Microsoft would feel like they could put those CSS definitions behind their vendor-specific '-ms-' prefix (like Webkit & Mozilla have done) and go ahead and implement them.

    In fact, Apple released a new page today that shows off some of the new capabilities of "HTML 5", such as it is ;-). I'd love to see this stuff running in IE9! πŸ™‚

    http://www.apple.com/html5/

    Cheers,

    – Bill

  36. David says:

    @William J. Edney – Unfortunately Apple is using browser detection to stop non-webkit based browsers to access those demos. They are also soliciting the download of Safari to run them. I don't know why they can't follow the example of the IE9 Test Drive page and just let every browser take a shot at it whether it works or not.

  37. Steve says:

    @avokicchi,

    Even if they told us what parts would be supported, it wouldn't stop the flurry of 'you MUST support feature y' comments!

  38. Klimax says:

    @David

    Simply use devtools or fiddler to change browser detection…

    But I agree,it is bad idea on their part.

  39. Stifu says:

    @David: "Unfortunately Apple is using browser detection to stop non-webkit based browsers to access those demos."

    Haha, boasting about how HTML5 is a standard and all that, yet blocking other browsers? How ironic.

  40. Wurst says:

    @Stifu: If you look at the source code it seems like they have originally intended to support other browsers as well (see for example the prefix list in "showcase.js").

  41. lem@webscannotes.com says:

    Great job! It seems that the IE9 team is dead serious about HTML5 video – GPU hardware acceleration would be most useful for HTML5 videos. C’mon, how often does one see a fanciful and confusing user interface? haha πŸ˜‰

  42. __hAl__ says:

    The flickr photo viewer gets stuck in IE9 when I enter a search phrase.

  43. Pierre. says:

    I do get 60fps with Safari 4 on the Mac.

  44. Aaron says:

    I don't use Opera normally except to test websites for clients, but I tested Opera based on the comments, and shame on whoever decided to leave it out.

  45. Blah says:

    Yes, shame on them for only testing on Windows and on browsers that a non-trivial number of people use! They should waste their time testing on platforms they don't compete on and against browsers that have virtually no users.

  46. Radek says:

    I'm really excited about IE 9, Can't wait for new build (hoping this time it will be with UI). Keep up great work!

  47. Andrew says:

    You know, a 3D Flickr photo viewer is great, I'm glad you guys are spending time on that.

    What would be even MORE awesome is if you would find a way to solve the compatibility issues around HTML5, an issue that is holding the entire industry back and where Microsoft and your team could make the critical difference.

    Many features of HTML5 are ready to go and, aside from minor issues, the compatibility between Webkit and Gecko browsers are not that great at this point. So many people are waiting for IE9 to arrive and thinking that it will be the key moment for HTML5 adoption on a large scale. The previews you guys put out engendered a lot of good will and high expectations that IE could finally reach its potential. Except they are forgetting something big that could be a bigger "anchor around our necks" than IE6 ever was…XP users.

    I don't think people realize how damaging the decision to not support XP users in IE9 is going to be in the long run. 60+% of the world's computers are still running XP according to some estimates. If IE on XP never gets HTML5 support, it will slow down the HTML5 process for years as we wait for all those people to upgrade to Windows 7. And you will be a decade older before your Flickr photo viewer app will be feasible for a truly mass audience. Instead the web will likely fragment as some smaller companies give up on supporting older users while others hold off on implementing new features.

    I don't know if it's Microsoft's strategy to try to steer HTML5 into the iceberg or just short-sightedness going on, but it's a looming problem and you guys are the ones who could fix this situation.

    And if I'm wrong about the practical effects of this, please explain how. How soon does Microsoft expect Windows 7 adoption to move to the point where an IE9-centric, HTML5 approach is viable for a mass audience? Why not just fix the problem once and for all by, if not changing course with IE9, patching the problems in IE8 by adding in at least a basic level of support for things like CANVAS, SVG and VIDEO?.

  48. Opera says:

    In my own tests of Flickr Explorer, Opera performed the best without sacrificing image quality whereas IE9 came second on my Intel X3100 graphics laptop with Windows 7 32-bit.

  49. Jote says:

    Leaving out Opera is ridiculous. I just ran the Flickr Explorer demo on my Asus 1005 HA netbook and I got 30-50 fps under Opera 10.53.

    Shame on you, Microsoft πŸ™

  50. Klimax says:

    Maybe they left out Opera due to Opera's actions against MS.

  51. Tom says:

    Microsoft feel ashamed because Opera is almost winning in term of Hardware Acceleration.

    Come on Microsoft, don't miss out Opera test.. show us your ball!!

  52. Lone Adult says:

    Maybe they left Opera out because… the set of people who actually run Opera is microscopic? And they all seem to hang out on this blog to say "why are you excluding us?"

  53. Prior Semblance says:

    I remember they showed Opera in another one of these comparisons and had to explain how Opera basically "cheated" by lowering the image quality.  Maybe it's something similar here and they didn't feel like explaining it for the minority of people who care.

  54. Mitch 74 says:

    @Prior Semblance: the 'cheater' was Firefox, which used (still does?) point sampling instead of filtering on CSS images, making resizing images much faster with lower visual quality.

    Opera's problem is not one of market share: it's that it settled, AFAICT, on an older form of hardware acceleration: video acceleration. In short, Presto computes the page in central RAM using the CPU alone, and then blits it to screen using a video output – which is WAY faster than GDI (unaccelerated Chrome and Firefox, IE 8) and compatible with Windows 2000/XP (unlike IE 9's Direct2D).

    Which may demonstrate that using derelict DirectDraw over brand-new-but-gotta-ditch-XP-to-get-it Direct2D might have been sufficient for a browser…

  55. Wurst says:

    @Mitch74: To be fair, Firefox lets you control the sampling mode through proprietary CSS.

  56. steve says:

    This is all very interesting but a total "Apples to Oranges" comparison.  Yes (ab)using the GPU to do all your intensive processing is a great way to improve browser performance but what happens when all the other browsers do the same? Are we going to see faster times across the board again, with IE still being the slowest because the core code of IE hasn't been fixed?

    I can't help reading all this about GPU powered HTML5 and think – well yeah, if you cheat by doubling your CPU power by "adding" the GPU to the mix of course you're gonna get better performance… but there's nothing stopping the other browsers from doing this and even shipping a version before IE9 goes RTM.

  57. Mord says:

    Amazing work Microsoft, your GPU-accelerated version still runs slower than CPU-accelerated Opera. I'm a web dev that has to use all five web browsers at work, so I'm glad if I don't have to spend quite as long as I do now making stuff work correctly (or at all) on IE but be honest with this blog and website. The HTML5 compliancy "test suite" was deceiving enough. You've tested with Opera thus far, why leave it out for this? You have a good IE on your hands, don't blow it with jealousy.

  58. Mitch 74 says:

    I'll add a new gripe: this forsaken new blog just can't be used with Javascript disabled: even when you properly enter the results of the mathematical CAPTCHA, you get redirected to a page vaunting the new MSDN platform – and not only is the post not submitted, going back loses the comment's content ("don't break the back button", rings any bells?).

    @Wurst: you do well to point that out – and anyway, it isn't cheating since it's not written in any specification that bitmaps must be resampled with bilinear or bicubic filtering. That's why I put quotes around 'cheating' that first time.

    What cracks me up, is that using Chromium on Linux, results in frame rates that hover around 40-60 fps except on resizes (zooming in); would it be that this wonderful post doesn't detail how great IE 9 is, but instead emphasizes how much GDI stinks?

  59. Safari says:

    http://www.apple.com/safari/

    Safari 5 released, embarasses Microsoft's outdated, insecure browser in speed.

  60. Matt says:

    Yawn… Safari 5 is comically slow. 4 frames per second on Flickr Explorer. 6 frames per second on Browser flip. A SunSpider score nearly half as fast as Opera's. Boring.

    I had high hopes for their "reading" mode, but it doesn't really eliminate the advertising like it claims to.

    Apple needs to up their game.

  61. Matt says:

    In terms of security of browsers on Windows (vuln rate, architecture, etc), it looks like this:

    Chrome > IE > Firefox > Safari > Opera

    So they've got quite a ways to go there too.

  62. Mord says:

    @Matt: I wasn't aware Chrome had more vuln than IE? I don't have more numbers but Secunia, for one, shows less vuln and less open exploits for Chrome 4.x (zero for 5.x) than IE 8.x? Could you link me? I'd be very interested in reading more about that. Thank you.

  63. Matt says:

    Mord: You're reading it backwards. Chrome has fewer vulns, and an improved security architecture over IE8 (because of Chrome's hyper-restricted render process).

    chrome's "Real world" security may be lower than IE8, however, because it has been shown that Msft's smartscreen filter is much more effective at blocking evil downloads, and they've blocked hundreds of millions of them. There's data that suggests that more users get owned by this type of trojan/malware/fakeAV than thru traditional vulns.

  64. Alex says:

    You forgot Opera 10.53. It would be tasteful to include it in these kinds of tests because it is the only other browser utilizing GPU acceleration on Windows.

  65. Wurst says:

    @Matt: Try to run the browser flip test several times on newer WebKit browsers, you will see that the FPS will go up (be it that the browser doesn't crash like Safari 5 did for me).

  66. Opera fan says:

    Again, I hate ppl that forget Opera, it defeats all of it's opponents on this (and basically anything else), I get 60FPS running Opera 10.6x…. To everyone who hates Opera, and even never tried it (97% of you guys, proven by 3% market share) It is easily the fastest, most complete and most customizable browser out there, it is just so lame to never show the browsers that gives the competitors nightmares. But I gotta admit that it works, because almost no-one gets to experience Opera, and for IE9, Microsoft shouldn't even bother trying any more, whit the "blessed" browser choice screen, which will hopefully, finally give Opera what it deserves!

    And I just know that whatever it would cost me I don't even try to make my sites compatible whit IE, no matter what version, maybe they could just follow Opera's example and follow "the rules" of the web, instead of being so arrogant in trying to make their own "rules".

  67. John says:

    Mozilla and Google are still dedicated to supporting hardware acceleration of XP, so it's still faster in my and 60% of the world's eyes.

  68. r414 says:

    Opera’s performance on IE9 speed demos >> zibin.tehais.com

    "

    IE9 preview has been making wonderful improvement. It started first with web standards improvement, and now speed progress. Though far from release and still a non-production, it is interesting to see from IE’s own demo that they are beating FF, Safari and Google Chrome.

    Friends at IE might have been busy that they missed out on testing Opera. Still it’s heartening that in their demo, Opera icon was used as a test image.

    "

  69. Matt says:

    r414: Read the comments on that blog and you see how he lied.