Enabling Professional-Quality Online Video: New Specifications for Interoperable Captioning


Video captioning on the Web is about to get significantly better, putting in place another critical building block for enabling professional-quality online video delivery and playback. To achieve the experience of broadcast television, the Web needs to provide captioning capabilities including word highlighting, tight synchronization between captions and spoken word, flexible caption positioning, caption styling, and caption animations – all of which are part of a recently published profile specification from the W3C Timed Text Working Group (TTWG). Microsoft has joined with content owners, video distributors, and device providers in developing the specifications to ensure interoperability and streamline closed caption authoring and delivery. For content and tool providers, the specification enables authoring of interoperable caption files for delivery to a wide variety of software and devices, while meeting evolving industry requirements.

Captioning is an intrinsic part of any professional-quality video experience—and the impact goes far beyond enabling the hearing impaired to gain full access to the video content. Captions are turned on in loud environments, such as at the gym. Captioning is used to help us all understand foreign language films. Of course, I’m surely not the only person who turns on captions to better understand difficult accents in movies streamed on Netflix.

In February 2013, Microsoft joined industry stakeholders in the W3C Timed Text Working Group (TTWG) to deliver the TTML Simple Delivery Profile for Closed Captions (SDP-US) profile specification. This new profile was created with input from media industry experts from DECE, SMPTE, EBU and industry players including Adobe, MovieLabs, NBC Universal, Cox, Apple, Netflix and Microsoft.

SDP-US is based on Timed Text Markup Language (TTML) (a caption interchange specification that has been used in the professional video industry for years) and clearly defines key caption format features like layout, style, timing and content requirements, enabling content authors and tool providers to easily create interoperable caption files that meet evolving industry requirements. SDP-US will help streamline creation of closed captions to deliver media content to the Web across a wide variety of software and devices – in a browser like Internet Explorer, on devices such as Xbox or through applications built using the Microsoft Media Player Framework

Internet Explorer was one of the first browsers to include early support for HTML5-based video captioning via the <track> element with TTML and WebVTT file formats.  Since then, we’ve heard clear feedback from content authors: They need an interoperable, simple and full-featured captioning solution that meets emerging requirements for browsers and other software on Internet-connected devices.  SDP-US will meet this need by defining a streamlined set of captioning capabilities for the HTML5 <track> element.  Developers will be able to add captioning to an HTML5 video by providing a caption file that contains a styled text representation of the video dialog or actions and by using the <track> element to render and display the contents of that file. 

To illustrate some of the power of the new SDP-US captioning specification, here’s an example of how Internet Explorer renders captions with a default plain text style and position:

Captioning in IE10 always looked like this

With SDP-US, caption authors have much more flexibility in caption positioning and styling, as shown in the below examples:

SDP-US allows captions to be placed anywhere on the screen

It allows caption text to be emphasized and colored

And caption regions can be filled with color

Professional-quality online video is a forthcoming reality, enabled by emerging Web specifications and powerful content delivery infrastructure. Captioning is an important building block for enabling professional-quality video, and Microsoft is actively working with industry partners to enable rich captioning experiences. If you are working on Internet video, we invite you to review the new SDP-US profile, join the industry discussion, start considering how your caption content can adapt to SDP-US, and let us have your feedback.

– Sandeep Singhal, Group Program Manager, Internet Explorer

Comments (14)

  1. Dennis says:

    Too bad we can't watch any media uploaded on Wikipedia and any Wikimedia's website on IE, because all their media (audio and video) is only supported in open format.

    Why can't Microsoft adapt more open formats, such as Theora Vorbis and show some maturity in the amateur format war?

    Its an open format. Implement it in next release and you are all set! What are you guys waiting for? More bad rep, or marketshare drop?

    Oh wait, Microsoft has to follow Apple on this one! Why Microsoft wants to drown itself after Apple? Do they care about your products or reputation of your products?

    Cmon already… these days, people don't have patience for stories. They open IE, go to google, install chrome and never open IE again. Same treatment happen to Safari on Mac, in favor of Chrome…

  2. Jaimes says:

    * We're still waiting for the Textarea fix for all versions on IE10 (metro, desktop on both Win7 and Win8).

    * We're still waiting for the on screen keyboard fix for Windows 8 desktop on touch screen tablets/surface.

    * We're still waiting for the IE Blog comment form to be fixed.

    * We're still waiting (but are glad to hear the fix is on the way!) for the return of the Start button that was removed in Windows 8 but shouldn't have been because so much functionality was completely lost

    Please don't bother posting more articles on how you are not supporting open (non DRM-encumbered) audio and video formats until you've at least fixed the textarea and keyboard bugs!

  3. fred says:

    @Dennis: Because as soon as Microsoft commits to a format such as Theora, or whatever, the "Open all the things" crowd will decide that they don't want Microsoft to be compatible and will come up with something else with dubious advantages over current technologies. They'll implement it while it's still in Alpha and then laugh at Microsoft for being behind. Rinse…Repeat…

  4. George says:

    @Jaimes: Are people still griping about the start button? Really? I HATE needing to use it at work. It's clunky and slow and difficult to navigate compared to the start screen.

  5. Jorrit says:

    Never put 'Simple' in a standard-name… It always proves to be too simple and becomes bloated later on… See SMTP and SNMP.

  6. Dennis says:

    @fred, I agree with you. This is how Google roles. The "real" open source community don't take Google as open source group and identify them as "some company selling your privacy for money and have other evil agendas".

    OSS has a legacy of supporting Microsoft products and getting benefit from them as well. OTOH, Google is merely doing everything in its power to destroy Microsoft.

    Open Gmail on IE9 or IE10 and try to download attachments, drag drop attachments and different stuff. It won't work! Then open outlook.com on Chrome and IE, try to do the same, you won't feel any difference. I know what happened with Google's SPYD protocol and the one before it, as soon as Microsoft supported their protocol, the moved on to a new one.. Then Chrome developer's tools has all Mobile platforms for testing responsive design except… of course.. Windows Phone! Windows Live Movie maker supports Android and Chrome extensions for Video conversion. Now they are at the verge of dropping support for apps in Windows 8 and Windows Phone 8 (like they destroyed MetroTube WP app by denial of service for couple of months… MetroTube has THE BEST and most rhetorical interface for youtube till date).

    Some people confuses Google's theory of openness (ripping off users privacy) with Open Source Society… but those are not the real OSS people. The guys at Redhat and contributors of Linux and other open projects recognize the contribution of Microsoft in the domain of Computer Science and Software Engineering…while Google is born to deny everything that Microsoft developed till date. To me its not even a necessary evil. World is a better place without fuckin Google!

  7. @George – the Start screen tiles vs an organized Start menu? I'll take the start menu *any* day! especially when the start screen doesn't contain everything I need and especially when it only fills 1 of my 3 monitors! why must I suffer a 66% reduction in desktop space because MSFT wants to add a bunch of oversized boxed for my non-touch screen environment?

    @Dennis – they are not supporting Windows Phone 8? and this surprises you somehow? who does support Windows Phone 8? As for the codecs it is totally understood… there should be at least 1 fully open source audio format supported by *all* browsers and 1 fully open source video format supported by *all* browsers.  Sadly until there is HTML5 Audio/Video just won't take off because we need to support additional formats to support IE browsers.

    @Jaimes – the most frustrating thing about this blog is the comment system. I finally gave up and registered because it was such a pain to deal with and no one at Microsoft seems to care to fix it.  However even with being signed in it still fails every now and then.  If I lived in the states I'd start a kickstarter fund to get some attention on the issue… maybe a full page Ad in the Seattle Times would get some attention?

  8. Fleet Command says:

    Finally! A standard for close captioning. However, I have three concerns:

    1. It will all be for nothing if Windows Media Player does not support it as well. We download podcasts, you know. It would be horrible to have captioning online but not offline.

    2. It will all be for little value if Wikipedia is not covered. Wikipedia and it sister projects are now on of the top ten high-traffic websites of the world.

    3. Internet Explorer is an exotic web browser. If someone asks "Can I install the latest version of IE on my operating system?" the answer is almost always "No". Even when the answer is yes, the installation packages (plural) are huge, around 199 MB. So before the users have time to get excited about one feature such as Timed Text, they have to address much larger disappointing aspects.

  9. Dennis says:

    @steve_web. what is your point and wtf you are talking about? There are over 200 thousand apps available for WP8 and tons of famous games/apps developers building for WP8. Google has recently DROPPED the support for WP8, meaning they had apps which will not be available anymore. BTW, besides WP, I mentioned IE (outlook vs gmail on ie and chrome.), Windows Store and other Microsoft products which Google barely support and recommend its hypnotized users to not use them.

  10. ieblog says:

    @Jaimes, steve_web – I apologize for the frustration caused by the blog comment system and would like to get this fixed.  I have not been able to repro the issue myself.  Can you please provide me with the following info?

    1. Nature of the problem – (Intermittent or Constant? Just started or has never worked?)

    2. Detailed repro steps

    3. Environment info (OS, Browser and Version)

    4. Any error message you see

    Also feel free to contact me directly at ieblog@microsoft.com.

    Thanks,

    Jon Aneja

    Program Manager, Internet Explorer

  11. Jens says:

    Good News.

    Amazone Video use Silverlight now :)

  12. @ieblog says:

    If you post a blog post (without having an account!!) more than 10 minuten (or something like that) after this page was rendered the post disappears. It is not submitted to the blog.

    This happens especially if you read a long blog post or even worse you compose a long post yourself.

    That would not be a big problem if the post was retrieveable with the [back] button of ALT-arrowleft. However that is also impossible. This blog does not retrieve your post by going back. Normal blog software does retreive your post on a post failure using the back button.

  13. Randall says:

    Second @ieblog above. The problem happens if there's a longish wait between when I come to the page and when I submit comment. I've gotten the habit of copying whatever I type to the clipboard and reloading before submitting.

    If it never happens to you however long you wait, maybe users are only affected if they aren't logged in, like me, or something obscure and configuration-related, like our Internet connections migrating between IP addresses or something to do with what browser we use (Chrome 25 on Ubuntu for me). Or, finally, maybe it used to be a problem but isn't so much anymore, and now we're just being fooled by the moderation delay.

  14. Mikelon says:

    By the way, is there any chance that the version of MSHTML in IE 7 will generate strict correct code? Quotes on attributes, lower-case tags, well-formedness etc etc?

    http://gtaonlineargent.com/