HTML5 Video Captioning


The promise of HTML5 is a Web that works for everyone. Media accessibility through captioning is an important part of that promise and an area that is still emerging through standards work. Using the proposed HTML5 track element, developers can add captioning to HTML5 video by pointing to a caption file that contains a written representation of the dialog or actions in the video. Once the standards for captioning converge, there will be less need for external add-ons to publish accessible video content.

Still image from a video showing captions

The following HTML fragment shows how the track element works:

<video>

<source type=”video/mp4″ src=”video_file” >

<track src=”captions_file” label=”English captions” kind=”captions” srclang=”en-us” default >

<track src=”descriptions_file” label=”English description” kind=”descriptions” srclang=”en-us” >

</video>

The W3C HTML5 specification allows for multiple caption formats. Internet Explorer 10 in the Windows Developer Preview supports the <track> element, but does not download or display any caption files yet. We use this implementation to prototype how different caption formats can be supported.

Last year at the W3C TPAC meeting, media accessibility was discussed with a proposal for the accessibility requirements that HTML5 should support. Recently, a Web Media Text Tracks Community Group was created to further work on captioning solutions for the Web.

Even though we are not yet at the stage of a final HTML5 spec, it is stabilizing. In order to provide better feedback on <track> and caption file formats, the Internet Explorer and TwC Accessibility teams partnered to build a prototype that uses the element to display captions and descriptions on a video.

We wanted to test a scenario were multiple formats are used to gather more feedback, so we built the demo in such a way that both TTML-1.0 and WebVTT can be used. WebVTT originated from W3C discussions last year after a need for simple caption authoring was identified. TTML is already an established standard for full-featured video captioning and supported in Adobe Flash and Microsoft Silverlight. It is used on Netflix, Hulu and other sites that display broadcast content.

Screen shot of the prototype that displays TTML and WebVTT video captions and descriptions

The prototype uses the video and track elements to show how TTML and WebVTT formats can be used to enable captions and descriptions on a video playback. The prototype supports all HTML5-enabled browsers.

You can interact with the prototype at the HTML5 Labs site. Use View Source or your browser’s developer tools to inspect the Javascript that parses the TTML and WebVTT formats.

As always, we welcome your feedback.

—Frank Olivier, Senior Program Manager, Internet Explorer

Comments (16)

  1. hart says:

    Please – made foolscreen for HTML5 video

  2. devdev says:

    This is great! Only Firefox supports the track element but also no caption file downloaiding and displaying.

    Will IE support webvtt or ttml? pleaase do webvtt because every other browser will be.  

  3. devdev says:

    or please do both because flash does ttml

  4. Richard says:

    How about you guys just read the goddamn article?

    "so we built the demo in such a way that both TTML-1.0 and WebVTT can be used"

  5. John Dyer says:

    Nice work on the TextTrack implementation and WebVTT parser (http://www.interoperabilitybridges.com/…/jquery.msft.video-access.js). Will this be open sourced as a library or just added into IE10? I'd love to use something like it in http://mediaelementjs.com/

  6. Full screen, full screen, full screen!

  7. g says:

    Could you change the style slightly to avoid using only mozilla- or webkit- prefixed CSS attributes? It looks ugly in non webkit or mozilla based browsers.

  8. Brendan says:

    Will there be fallback compatibility support for IE6 ??

  9. Mario says:

    Hello Internet Explorer Team, I have a question that needs answering fast please answer.. Internet Explorer 9 won't show the images on the bottom of the web page at play.clubpenguin.com/index_small_screen.php  I cleared my cache and cookies about 2 times.. on my other computer that is a desktop the images shows up but not on my laptop computer for some reason.. 🙁 please help

  10. Mario says:

    Do i need to clear ActiveX filtering and tracking protection cache also?

  11. Nelson says:

    Dear IE Blog – Please for the love of god fix your index page not to show the FULL ARTICLE for all your recent posts!  Not only does no other sensible blog do this but there are major issues with the current setup.

    I just did a full no-cache load of the main index page which:

    loaded 7,442.1K of data!

    took 14.358 seconds to fully load and render

    and consumes 63 "pages" full of content (vertical height) on a widescreen desktop LCD.

    That is COMPLETELY UNACCEPTABLE by ANYONE'S standard of usability and performance.

    Worse yet, I sometimes read the IE blog on my iPhone (or other mobile device) – this volume of data is absurd considering all that anyone is actually looking for on the "home" page is a LINK to the article that they might care about!

    To pour salt into the wound – I have to use the comment system here to leave a response – which I know full well will FAIL to save my POST so I have to copy it to the clipboard first, perform my first submit (the one that will fail for sure), then reload and hope the 2nd or 3rd submission will work.

    Please for the love of god make the home page contain *****STUBS***** for each article that we can link to and change the "post" button in the Leave a Comment section to the following 100% known-to-work-doesn't-use-any-hacked-up-asp-post-back-garbage-that-doesn't-work-ajax-mess HTML.

    <input type="submit" value="Post"/>

    This mere 35 character code addition will solve a bug that this blog has had for over 5 years.

  12. Since commenting is closed on old posts, like blogs.msdn.com/…/testing-multiple-versions-of-ie-on-one-pc.aspx , I want to ask Microsoft to stop being evil, and allow http://www.spoon.net/ to host Internet Explorer versions, for testing purposes.

    Forcing me to install 3 virtual machines to test IE8, IE7 and IE6 on my Windows 7 laptop with IE9 is evil. The 3 VMs use 29 GB of disk space, and I only have a 80 GB SSD.

    I wouldn't have commented here, if the VMs would have worked properly. But the latest Vista virtual machine crashes with blue screen as soon as I visit two or three pages with IE 7.

    I uploaded the crash dumps to http://www.mediafire.com ; the small file contains minidumps and associated files, and two blue screen screenshots; the two large files are two full memory dumps, if anyone is interested.

    I downloaded the VMs yesterday from http://www.microsoft.com/…/details.aspx (Windows Virtual PC VHDs for testing websites with different Internet Explorer versions [Last Updated: 8/16/11]).

    To conclude, please allow Spoon.net to host Internet Explorer.

  13. Alec says:

    Although I appreciate the IE team making strides on video HTML5 implementations, the underlying problem is a lack of a uniformity in the file formats among browsers. Please add support for Ogg Theora and WebM

  14. Sean Jenkin says:

    @Nelson – You can control how the blog home page is displayed by clicking on the Excerpt icon next to the Sort By options. That'll reduce your page load. As to comment submission, the dev team is actively working on fixes for the comment submission at this time. We hope to have a patch out soon.

  15. Fermant says:

    On this W3C test w3c-test.org/…/paused_true_during_pause.html

    IE passes the test but continue the playback while it should remain at the paused state. Other browsers don't show this behavior.

    OAN, IE fails all the tests laid by Apple w3c-test.org/…/Apple

  16. BrianMB says:

    The best demo and suggested implementation of <track> I have seen.

    On an added note, you guys are really in high gear with IE10.  I feel like almost all needs will be met by this version.  I would like WebRTC, WebGL, access to OS-level functionality via the browser, et al.  However, I don't feel constrained by IE when I see version 10.  That is a really remarkable difference.

Skip to main content