Comparing Hardware Accelerated SVG across Browsers with Santa’s Workshop


Internet Explorer 9 fully hardware-accelerates the entire Web platform. Different browsers take different approaches to hardware acceleration. These differences result in different levels of performance. Today we look at Santa’s Workshop, one of the entertaining Test Drive demos we released during the holiday season, and see how the different approaches to hardware acceleration benefit consumers and developers.

Pack as Many Presents as Possible

The goal of Santa’s Workshop is to pack as many presents as possible in real time. A scoreboard at the top of the screen keeps track of how many elves are working, how many presents have been packed, and a presents-packed-per-minute working velocity which we heard is how Santa evaluates his elves during their review cycle.

Screen shot of Santa's Workshop IE Test Drive demo

Santa’s Workshop is a performance demo written entirely in SVG and JavaScript. The more efficiently your browser can animate elves moving around on the screen, the more elves will appear and the faster they’ll be able to pack presents. Santa’s Workshop uses many emerging HTML5 and SVG patterns, including SVG DOM manipulation, applying transforms to groups of SVG elements, and playing background music through the HTML5 audio tag.

The demo keeps track of the frames per second (or number of times this loop is processed per second) and uses this information to determine when to add or remove worker elves. While the frame rate is above 55fps (meaning we’re running at close to real-time animation on a 60hz monitor), another elf is added. When the frame rate falls below 35fps, an elf is removed. The goal of this algorithm is get as many elves working at once as possible while maintaining a good experience.

Performance of Santa's Workshop demo in for browsers

Differences Between Browsers

Let’s take a look at how the different browsers use machine resources to when running the Santa’s Workshop demo. The below results were generated using the Windows Performance Toolkit which can be downloaded from Microsoft, and to continue to show the benefits of hardware acceleration across a broad range of hardware, these results were generated from a Dell Laptop (2.26 GHz Intel Core i3, 4GB physical memory, NVIDIA GeForce GTX 260M, 250GB 7200 RPM drive, Windows 7).

With Internet Explorer 9 RC, Santa’s Workshop is able to achieve 50 elves and pack 238 presents per minute. The elves smoothly animate across the screen in real time and the contents of the scene are rendered correctly. Below, in red you will see that Santa’s Workshop uses one full core of the CPU while the demo is running. Today’s Web platform is single threaded meaning the browser can’t distribute this work across multiple cores. In blue you will see where the GPU is used to render the scene to the screen. GPUs are massively parallel and extremely efficient at graphical computation. And in green you will see where the screen is updated after the GPU has finished rendering the scene.

Performance of Santa's Workshop demo in Internet Explorer 9

Next, let’s look at the latest Chrome build available at the time of this writing—Chrome 11 Canary with hardware acceleration enabled. With Google Chrome 11 we can only achieve 15 elves packing 71 presents per minute. When we look at the CPU, GPU, and updates we see that Google Chrome 11 is essentially using the same amount of machine resources as Internet Explorer 9. This may be surprising at first considering Chrome 11 can only pack 30% the number of presents as Internet Explorer 9.

Performance of Santa's Workshop demo in Chrome 11 Canary

Firefox 4 (Beta 12) now leverages hardware acceleration by default, however this most recent version of Firefox was only able to achieve 1 elf working and pack 4 presents per minute. In addition, the frame rate quickly fell to 13 frames per second. When we again look at the CPU, GPU, and updates charts you can see that Firefox’s usage of the GPU is very sporadic. The majority of the graphical computation occurs on the CPU and only occasionally do you see computation occur on the GPU.

Performance of Santa's Workshop demo in Firefox 4 Beta 12

Safari 5 did a bit better than Firefox 4 and was able to achieve one elf working and pack four presents per minute at a frame rate of 20fps. Here’s how Safari uses the machines resources to animate the one elf around the screen.

Performance of Santa's Workshop demo in Safari 5

As you can see, all four browsers were using comparable machine resources. They fully used an entire core of the CPU and performed very similar amounts of GPU computation. Yet the amount of work each browser was able to perform using these resources and the user experience varied significantly.

Browser CPU GPU FPS Elves
Working
Presents Packed
per Minute
IE 9 100% 20% 56 50 238
Chrome 11 100% 16% 41 14 71
Firefox 4 100% 4% 13 1 4
Safari 5 100% 10% 20 1 4

Full Hardware Acceleration is the Difference

With Internet Explorer 9 we’ve fully hardware accelerated the entire Web platform. Different browsers have taken different approaches to hardware acceleration. They’re choosing to accelerate only pieces of the Web platform, building abstraction layers between the hardware, and using underlying machine resources with different levels of efficiency. Our approach is simple though – we’re going straight to the hardware.

Santa’s Workshop is a fun example of a hardware accelerated SVG driven experience. We enjoy building these goofy demos and stressing what’s possible with hardware accelerated HTML5. And considering Santa’s Workshop received over 100,000 unique views during the holidays, it seems you have as much fun playing with these demos as we have building them.

We hope this post has been helpful in understanding some of the different performance characteristics across the browsers and encourage you to start thinking about what’s possible with fully hardware accelerated HTML5.

—Seth McLaughlin, Program Manager, Internet Explorer

Comments (47)

  1. Miguel Web Developer says:

    OK.. We get the message about acceleration. Always it!!! And about http://html5test.com/ ? When is IE9 start to pass Chrome?

    Chrome: 282

    IE9:            32

  2. Miguel Web Developer says:

    html5test.com/results.html

    Microsoft Internet Explorer 9 RC  – 130

  3. BS says:

    HTML5test.com does not test functionality, what's the point

  4. Ashen says:

    Even the mobile browsers have higher scores than IE9

    html5test.com/results.html

    Atleast try to beat them.

  5. mbreslin says:

    Chrome – dev: 38 elves, not sure how you get 17 elves on canary, something is up.

  6. mbreslin says:

    Sorry I misread you actually state 14 elves, to my 38. I thought for a moment there must be some reason there is such a difference in your number from mine, then I looked at the domain I was at, figures.

    Embrace and extend, when that fails, just make *** up.

  7. Robert O'Callahan says:

    "Full Hardware Acceleration is the Difference" … well, no it isn't :-). I just did some profiling of Firefox and we spend less than 8% of the time painting. Almost all the time is in the DOM manipulation of transform attributes, so we must be doing something stupid there, which we'll have to fix. Nothing to do with "hardware acceleration" though.

  8. Jace says:

    @Robert O'

    Wait, if you're doing something stupid, were you actually 'hardware accelerating' when you ought to have been?

    Did the "stupid stuff" prevent feeding of the 'hardware acceleration' pipelines?

    In the end, no matter how stupid Firefox code is, it's obvious that one browser used compute resources efficiently…

    Good thing for competition though, maybe other browser vendors will get the message, remove their 'stupid' code, make the world greener and usher in peace and prosperity.

    Sorry, left yourself wide open, couldn't resist 😀

  9. Boris says:

    @Jace

    > Did the "stupid stuff" prevent feeding of the 'hardware acceleration' pipelines?

    Yes, precisely.  

    But the point is that the "stupid stuff" is not work that can be done on the GPU.  It's just work that needs to not be done at all in order to make this fast.

    Or put another way, it doesn't matter whether you hardware accelerate the drawing of this testcase in Gecko, because the gating factor is something completely outside the drawing path.

  10. Robbie Alexander says:

    Give them a break over the html5test.com things take time to implement and most of those elements aren't even finalized yet or implemented by others. I'm sure MS has prototype code for many of those elements in work in future versions of IE but for this release they have focuse on the key elements such as canvas and video like others which is good! They also have made a major push for hardware acceleration as shown here which others have neglected because it is difficult and the results have been great for all parties! This is a very respectible browser release by MS and they deserve credit for it regardless of your view towards their corporate strategy.

    Kudos MS, I look forward to the day IE9 becomes a pushed update the the masses! :)

  11. Will K. says:

    The hardware acceleration is nice, but really does not add anything for standards support..

    As others have indicated, the rather low level of HTML5 support is quite worrisome.

    As the common baseline for all browsers, if IE9 has poor support, then developers will need to hold back on HTML5 adoption.

    While it is already a RC and there are rumors of imminent release, I guess it is too late for much more changes.

    IE9 is a large step forward over IE8, but only so much could realistically be done.

    So please start talking to us about IE10.

    What is the general release schedule? You need more frequent releases such as every 6-9 months.

  12. meni says:

    First, well done on h/w acceleration. It really shows that there are some talented developers are working in Microsoft.

    Second, a Question for the team:  Why is WebGL getting the cold shoulder?

    a) It's not in HTML5, and we are concentrating on HTML5 tech only

    b) It will be too hard to implement it. Developing something like Google's ANGLE project will take a long time.

    c) WebGL, being in the OpenGL family, competes with DirectX, and as a matter of pride and politics we will never touch it.

    In any case, Microsoft is the only vendor not implementing it.

  13. jj says:

    I get 6 presents per minute in both Firefox 4 and Safari 5 on my five years old laptop, so it's surprising to me that you only get 4 for these. If I understand the performance metrics correctly. (MacBook Core2Duo 2.0 GHz, Mac OS X 10.6.6)

  14. ChrisTX says:

    @meni

    d) WebGL's first stable specification was released 3 days ago

    e) WebGL is not a W3C standard at all

    f) WebGL isn't even undisputed.

    @Ashen

    Tell me when any of these browsers getting scores for Web Sockets also implements a current web socket revision and not -00. Just one example of how crappy that test is.

  15. I'd like to thank the IE team for working hard to resolve the performance issue of Internet Explorer. For a long time now, Internet Explorer needed a performance boost.

    But allow me to remind the IE team that it is not enough. Internet Explorer's performance over the network also requires improvements. Experience tells me that Chrome, Firefox and Opera operate much better when there is network traffic congestion or over slow or very slow network. Pleas bear in mind the networking performance.

  16. EricLaw [MSFT] says:

    @FleetCommand: We've actually made major network performance improvements in IE9 RC. Stay tuned– we'll have a post discussing these on the IEBlog in the next few weeks. Thanks!

  17. Peter van Dam says:

    Nice article and I really believe that IE9 does the best thing in hardware rendering. But one thing troubles me….

    "…. Our approach is simple though – we’re going straight to the hardware."

    I guess you mean with this sentence everything goes straight to the GPU. Because thats what you do different.

    But why the heck is just 20% of GPU used and 100% of a processors core. You're not fully using the GPU yet, else the CPU wouldn't be at 100% all the time.

    So what exactly is made different? You say others use layers and different parts to split things between cpu and gpu. Your telling everything is done by GPU, so why is the CPU still used 100%???? So I see this as doing exactly the same… So more clarification here would be nice….

  18. zzz says:

    Is anyone able to easily select with a mouse the text "presents per minute" by clicking and dragging somewhere in that sentence while the demo is running? I am able to select the other 2 texts except the "Santa's Workshop". This is on preview 7 as I'm not replacing IE8 until the fuzzy, too small at 100% zoom text and the pop-up bars that used to be at the top below the tab bar are at the IE8 location again. (Imagine if MS hardware moved the space bar/key on top of the keyboard after everyone learned to use it at the current location…)

  19. zzz says:

    s/are at the IE8 location/are like in IE8/

    If you are going to keep the new location of the browser related pop ups at the bottom then I'd like to see some argument on what users will gain from that change and having to learn a new way to use IE and that you aren't going to keep changing it again. Also even if IE in future works in a mobile device, the mobile/touch experience should not drive changes to how the key+mouse desktop experience works.

    In my quick tests of the RC on another computer I found that the new location of eg. download pop up was such that I didn't even notice it and in the particular page I was using the pop up visually blended into the page much more so than the IE8 bar that slides from under the tabs.

  20. Bert says:

    Chrome seems to get slower over releases for this specific demo on my notebook.

    A slightly older chrome 9 got to 24 elves.

    Then I received the automatic update to chrome 10 and the number dropped to 13.

  21. Matthew says:

    As a developer I am amazed at how much IE9 has come along. I am keeping my attention on how the hardware accelerating between different browsers is coming along and I also keep going to the IE 9 Test Drive site. Thanks for making such a good resource available to demonstrate your browser and thanks for making it so that you can compare other browsers in it too.

    It's a much more mature approach than the one Mozilla have taken with their test site that uses a bunch of -moz- only prefixes without any other support for Webkit or Trident. That said Mozilla supports the history API and CSS transitions in their engine… *coughs* :)

    Thanks for some amazing work on IE9. One thing of particular note is the quality with which I have seen things implemented, this does get noticed by people.

  22. Pat says:

    Not on my machine.  ThinkPad 201.  IE9: 1 elf.  Chrome: 50 elves.

  23. EricLaw [MSFT] says:

    @Pat: I suspect you'll find (as I did) that your slow performance comes from running an outdated driver on your Lenovo. See blogs.msdn.com/…/ie9-benchmark-tests-and-gpu-fallback-to-software-rendering-mode.aspx for more details.

  24. Brianary says:

    We have to take these claims with a grain of salt, of course, since the IE Team tends to overstate things sometimes: http://www.webmonkey.com/…/ie9-leads-pack-in-html5-support-not-exactly

    Also, IE9 uses *NO* abstraction layer? Straight Assembly language, eh? That must be hell to maintain for each individual video card!  😉

  25. Jeffrey Gilbert says:

    You guys already wrote this stuff like 5 or 6 posts now. Are we really just waiting on more chest beating from the IE team or can we please get back to the relevant content that i subscribed to this for? I don't want to see stats sheets and full write ups every time you guys come up with a new hardware acceleration test. Let's get back to standards compliance and feature requests and giving the masses what they're asking for, please. There's still a lot that can be done here.

  26. Chris says:

    Just for reference, I get 22 Elves and 171 presents per minute on Google Chrome 11 for Windows XP on an eight year old PC without hardware acceleration, not bad for Chrome (http://goo.gl/6HAJM).

  27. frymaster says:

    brianary: it certainly uses less of one.  The downside, of course, is it doesn't work anywhere that doesn't utilise the vista/win7 graphics system, including linux, osx, and indeed windows XP

  28. Jeffrey Gilbert says:

    @frymaster I'm sure it keeps development costs down to not support other platforms. Whatever helps push this specific browser out of the darkages so I don't have to limit my development to supporting tech that died a decade ago, I'm for it.

  29. Josh says:

    > "The downside, of course, is it doesn't work anywhere that doesn't utilise the vista/win7 graphics system, including linux, osx, and indeed windows XP"

    This is really getting annoying.  It also doesn't support NT 3.5 – and why exactly should MS support a decade old OS?  Enterprises, end users, it is time to upgrade.  XP should die as quickly as IE 6.

  30. Henk Poley says:

    btw, the Elves benchmark needs to run for a while before it levels off to its final level.

  31. 8675309 says:

    they should suggest to people on the test site that it may require an video card driver update

  32. Björn says:

    "Straight to hardware" like no DirectX and no graphics driver inbetween? SCNR

    Fun fact: There is an implementation of the graphics API 'Cairo' indeed going straight to hardware, ie. without using OpenGL or therelike. It only works on selected Intel GPUs though and was also just intented for demonsration purposes.

  33. Tyler says:

    html5test.com/results.html please. Hardware acceleration is nice, but it isn't going to reduce the number of hurdles I need to jump throw in order to support your browser! Not supporting more of the spec up front means X number of years of hoop jumping. I'm sick of if(ie) {} else {}

  34. Anonymous says:

    I use Chromium 9 on a Linux system (were graphics performance in general is much worse than on Windows), and I can get 50 elves packing over 200 presents per minute. Doesn’t look like fair test results to me…

  35. Well done but says:

    I'm vey impressed that Microsoft of all companies is leading the pack here int terms of speed.  I won't get into the fact that IE could never come close to matching the other browsers speeds without tapping into the GPU and thus these numbers are not quite so impressive.

    The problem is that this is IE9. There are millions and millions of users stuck on a PC using IE6,7,8 that will never experience these speeds, nor the HTML5 features that IE9 finally supports.

    Worst of all these improvements came at a cost.  Bugs that have existed in IE for years or even a decade have yet to be addressed causing instability because developers can't use one markup because IE can't set innerHTML properly across all elements, can't retrieve the correct innerHTML on A.N.Y. Element at all, and still shoves all kinds of crud into the global namespace that shouldn't be there causing nasty collisions that shouldnt be possible in the first place!

    For reasons like these I'm finding the whole news about IE9 to be extremely lack-luster.

  36. Anonymous says:

    I just started up Windows Vista to compare… Actually, IE 9 RC produces LESS elves on my machine (43).

    My system: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz, NVIDIA GeForce 9600M GT (512 MB), 4096 MB RAM, latest NVIDIA drivers installed on both Linux and Windows.

    I don't know how you got your test results, mine are completely different!

  37. Anonymous says:

    BTW: Both operating systems are running on the very same machine and no Virtual Machines are involved. It's an Amilo Xi 3650 notebook. Just in case anyone wants to doubt my results…

  38. WizKid says:

    I would rather have my browser would be fast at scrolling then SVG elfs: weblogs.mozillazine.org/…/the_myth_of_ful.html

  39. LinuxRocks says:

    What are you talking about? On my office machine (Intel  DualCore , 2666 MHz with a NVIDIA GeForce 8800 GTX  (768 MB) running winXP32) with chrome 10.0.648.127 I got 42 elves @ 199 parcels/min. As others say above, your numbers aren't comprehensible.

  40. Vitaly says:

    Just updated my intel 4 express chipset graphics driver

    Animation became much smoother, but result is the same – 1 elv with 6ppm

    Chrome became better with 50ppm

    Before the update – every browser – 6ppm

  41. duro says:

    chrome 10.0.648.127 here at 277 elves ppm with 38 of the little buggers working … odd.

  42. Funneh says:

    For me Chrome has a better score in this benchmark under Linux than under Windows

    22 elves with 95 ppm vs. values similar to yours under Chrome

  43. tehk says:

    wow, a closed source web browser that was developed by a team of programmers that optimized the browser itself to perform a simple demo and then use the results from that demo to extrapolate the idea that they have the fastest hardware accelerated browser. not only can we not disprove this, but the fact that ie9 is much slower than firefox and chrome in so many other areas almost assuredly proves my assertions.

  44. tehk says:

    Also, I had 22 elves working that packed 240 presents. So, even though I ahd less elves, they packed present much faster. I would tend to think this is more efficient that IE. (Test ran on Ubuntu 10.04 / Chrome 11 / 6 year old Pentium D processor / Nvidia 9600 )

  45. tcs says:

    IE9 gave me 45 elves at 351 ppm

  46. Kedas says:

    Is there a way (registry setting or other) to change the 60FPS limit??

    I would actually prefer 30FPS

    30FPS doesn't look really different but saves a lot of GPU energy/temp.

    IE fish tank 1000Fish (60FPS) asks 181W

    IE fish tank 500Fish  (60FPS) asks 155W (that is  26W for, well, nothing!)

  47. Kedas says:

    Oh I forgot: 50 elves 266 packages per minute: 165Watt  (ATI 4670)