Delivering video content through Internet is one of the hottest technical topics. The increase of bandwidth and connectivity in combination with the numbers from the last big video event delivered like Olympics, demonstrate the potential of Internet channel for this type of content. In the 2008 Summer Olympics Games in Beijing, NBCOlympics.com attracted close to 6½ million users daily, who, on average, stayed more than 13 minutes per visit and spent more than 27 minutes when consuming video. In total, users watched more than 6 million hours of video content or 60 million video streams, which represent a 5-fold and 20-fold increase compared to the Torino and Athens games combined. Another big difference that the last Olympic on line demonstrate it is relative, not only to the number of users, but also to the viewing time. In the past, online video consumption was limited to “snippets” of video, now the prolonged viewing indicates that users enjoy the experience and find the image and sound quality acceptable.
Worldwide, billions of videos are viewed online every month. This type of trend are confirmed to a lot of service that came up on the market with success in the previous months like BSKYB channels offering or the new version of Netflix service and many others.
Setting up an online video service was a complex undertaking and fraught with difficulties, and the process of preparing, loading, and delivering multimedia content, combined with the serving of appropriate advertising, required a lot of manual work and did not really scale very well. In particular “delivering” video content at good or high level of quality through Internet connection is not a simple thing. A lot of details and technical issues are involved on this and build a solution for this and monetize the result is really complex.
In the last NAB conference Microsoft announce a first Release of new technology called Smooth Streaming based on IIS7 Media Services and Silverlight, which the scope to simplify and make a reality delivering video content at high quality level through Internet up to true HD for Live and VOD. You can see this web site from akamai an example of HD content delivered with smooth streaming and at this website you can found others technical information about this technology.
In the previous months before this announcement I had the opportunity to test and build a real production environment based on this technology, inside a new Web TV portal of RAI – Radio Televisione Italiana ( http://www.rai.tv ) . In the new version of this web site recently released there is a new area based on this smooth streaming technology with fiction, sport and junior videos. Content from RAI website have a good quality but is not HD content because the original produced video is not HD. Based on this experience I want write some post on this new fantastic technology that can permit to deliver uninterrupted video streaming to Silverlight clients at High Quality, via HTTP in VOD and delivers massively scalable Smooth Streaming for live events.
Smooth Streaming permit to solve one of the most significant limits in Internet video streaming connected to typical behaviors of the last mile for reproducing High Quality videos with high bitrates. When we think to the way that the content is delivered to the client, we have a lot of parameter that change every second during the client reproduction. In particular bandwidth and CPU power availability conditions change every second and causing a video glitch and buffering during the reproduction of the content, especially for the content at high level of bitrates, generating a poor user experience .
Before to go in deep in Smooth Streaming technology and in order to explain difference and advantage that come with this new streaming technique I want recap the typical way used to delivery video content through Internet. Before the smooth streaming you can use essentially two different way for delivering video content:
- Progressive Download
- Traditional Streaming (RTSP, streaming through HTTP, RTMP, etc)
Both approaches have pros and cons and smooth streaming try to combine the best of the previous techniques.
When you use Progressive Download to delivery video content you essentially leverage the capability of Web Server to delivery file.
The Web Server receive a request for a specific URL for the video content and the web server start to deliver the file to the client via classic HTTP.
Big advantages of this solution are:
- simplicity of the approach ;
- not required a specialized server;
- leveraging all HTTP infrastructure in terms of caching and distribution.
Problems connected to this solution are:
- Bandwidth controls;
- Security of content;
- Limited user tracking;
- Poor user experience (no seeking feature on classic web server);
- VOD video content only.
In general the most significant problem of this approach is about bandwidth consumption. In fact the web server delivers the video content to the cache of the client without any type of control or limitation leveraging all the bandwidth available.
You can improve efficiency of progressive download through system that can permit the control of bandwidth and offering control and seeking capabilities. IIS7 Media Services enabled this feature too called Intelligent bit rate throttling. This feature is available for the most used type of media content, and permit to reduce bandwidth consumption saves money on network costs by metering the download speed of multiple media file types as well as data. Intelligent Bit Rate Throttling accomplishes this by automatically detecting the encoded bit rates of 11 common media formats, such as Windows Media Video (WMV), Flash Video (FLV), and MPEG 4 (MP4), and then throttling the response to the client. Bit Rate Throttling and allows administrators to configure throttling rules for all file and MIME types.
With this technique you could obtain more efficiency for bandwidth consumption and simplicity in the delivery but remaining a lot of limitations. For example, in the case of the user put in pause the reproduction the download proceeds consuming bandwidth; in this way we can use of a single bitrates for reproduction or complex variable bitrates technique with poor efficiency , and remaining the problem illustrated before about the last mile and client reproduction, in particular for content with high quality and consequent high bitrates.
When you use classical streaming you use a specialized server to delivery to the client only the portion of the stream involved in the reproduction of the video in base of the command received from the client.
Big advantages of this solution are:
- Efficiency in control and delivery of content ;
- Simple way to track user activity;
- Good user experience (seeking, forward, etc);
- Good for VOD and Live .
Problems connected to this solution are:
- Required specialized services ;
- Not cache in HTTP CDN;
- Scalability required a specialized network .
In general the most significant problem of this approach is about scalability. In order have a large distribution of video content is necessary achieve a specialized infrastructure of server with a specific streaming services. Is not possible reuse the big cache infrastructure that we have in place on Internet today for HTTP . In order to delivery this type streaming, Windows Server offers a good platform through the Windows Media Services today available for Windows Server 2008.
When you use Smooth Streaming now available in beta with IIS7 Media Services , you have the capability to deliver media at variables bit rates to Silverlight clients over HTTP combining the advantage of streaming and progressive download and adding capability of adapt the bit rates at the real situation in the last mile in terms of bandwidth and CPU.
With the Smooth Streaming you deliver video content starting from a set of specific encoded file with different bit rates level. During the encoding phase, realizable today for VOD (with Expression Encoder 2 SP1 with adaptive streaming encoding profile) or for live using a specialized encoder from an encoders vendor, you obtain the source files to stream. You can choose a set of different bitrates and obtaining a set of N file one for every level of bit rates that you have selected in the profile and a xml manifest with the metadata for server and an xml file for client. You can choose at maximum of 10 different bit rates. The manifest files are used from the server and from the client to understand level of bitrates available for streaming and others information relative to the media content.
A typical Smooth Streaming media asset (presentation) consists of the following files:
MP4 files containing video/audio
- *.ismv Contains video and audio, or only video (one file ISMV file per encoded video bit rate )
- *.isma Contains only audio
Server manifest file
- *.ism -Describes the relationships between the media tracks, bit rates and files on disk . Used by IIS Media Services Server
Client manifest file
- *.ismc - first file delivered to the client. Describes the available streams to the client: bitrates level available, codecs used, video resolutions, markers, captions, etc.
For live streaming the same structure is created in the archive of event and client manifest is generated at runtime from encoding available. Origin Media Server can archive all live streaming and offer DVR capability to the client.
In every video\audio file (ismv , isma) you have a contiguous sequence of video chunks at fixed time length (default 2 sec) based on the ISO/IEC 14496-12 ISO Base Media File Format specification, better known as the MP4 file specification . There are actually two parts to the Smooth Streaming format: the wire format, and the disk file format. In Smooth Streaming, a video is recorded in full length to the disk as a single file (one file per encoded bit rate), but it's transferred to the client as a series of small file chunks. The wire format defines the structure of the chunks that are sent by IIS to the client, whereas the file format defines the structure of the contiguous file on disk, enabling better file management. The MP4 specification is perfect for this because allows MP4 to be internally organized as a series of fragments, which means that in Smooth Streaming the wire format is a direct subset of the file format. Every fragment is delivered to the client through HTTP and in this way can be leverage all the HTTP caching infrastructure of Internet.
For VOD and for Live the content are published through IIS Media Service using Windows Server 2008 and IIS7 with IIS Media services Extensions.
During the delivery of the content, the client, in order to reproduce the video, sends a set of specific requests to the server to download the video and audio chunks. In every request for a single chunk, the client specifies level of bit rates and timeframe required in base of the current condition of bandwidth and CPU and position in content reproduction. In this way smooth streaming is delivered with a set of progressive download of video and audio chunks.
The client send a set of HTTP GET request in REST style like this : http://smoothstreaming.com/title.ism/QualityLevels(1775000)/Fragments(video=840000000) that contains information about bit rates and timeframe requested for the chunk . This type of request is a typical HTTP request and is it possible leverage all the http infrastructure on Internet in order to cache this content to near as possible at the client and using a typical http edge offered from a CDN.
A specific part of code downloaded in the client contains the Heuristic Algorithms that drive the requests of chunks. The process start with the base request from the client at the client manifest file on the server in order to obtain the metadata that can guide the behavior of the client. This request have this format : http://smoothstreaming.com/title.ism/Manifest, IIS receive the request send manifest client in response. In the next picture you can see a typical client manifest response with the indication of the bit rates available for the content downloads.
The client manifest can contains more information about media metadata like : codecs, Text streams, Multi-language audio tracks, Alternate video and audio tracks (for example, multiple camera angles, director's commentary, etc.), Multiple hardware profiles (for example, a bit rate targeted at different playback devices) , Script commands, markers/chapters, captions
The process continue with the request of the first chunk video and audio, requested with the low bit rates available in the manifest with a REST style call described before . The IIS Media Service Module receive the request and use a server manifest file to determine the file to load in base of the request bitrates and provides to the right chunk of video. In the next picture you can see a sample of server manifest file with the indication of file for bitrates:
For the successive request based on download speed, the client can detect available bandwidth and requesting the right bit rates in base of the current situation of bandwidth and CPU. In this way the smooth streaming have the capability of adapt continuously creating an immersive User Experience. The process Work in a similar way for Live and VOD streaming and during the delivery of the video chunk through Internet, all the HTTP caching infrastructure of the network can be leverage in order to improve scalability in particular for Live Streaming. In the case of Live Smooth Streaming the Origin media server can archive the content produced from the encoder and offering DVR capability too during the Live Streaming.
There is a lot of advantage in this approach respect to traditional streaming technique. In traditional streaming, the client state is managed both by the client and the server. The server keeps a record of each client for things such as playback state, streaming position, selected bit rate (if multiple bit rates are supported), etc. While this gives the streaming server more control, it also adds overhead to the server. In terms of scalability another important limitation is about the fact that each client has to maintain the server affinity throughout the streaming session.
With the pull model in VOD or Live Smooth Streaming, the server is stateless, the client drive the process and has the responsibility for the state and this fact increase scalability and remove any single point of failure. Any client request can be satisfied by any server configured and the network topology can leverage a stateless infrastructure and can reroute the client requests to the best server. In this way we have a big improvement in:
- quality because all the decisions, resulting in a much faster response (for example bit rate switching) offering a superior UX :
- reproduction start fast because the client select the low bitrates and can download in parallel more chunks, potentially cached from the most near web server
- bitrates are continuously adapted at the real situation in the last mile in terms of bandwidth and CPU status
- requests can be served from different servers without affinity or persistent connection improving scalability and fault-tolerance
- scalability because you can leverage a simple, efficient and load balanced HTTP infrastructure to deliver video content
- cost because you can leverage an HTTP CDN with generic web server with low number of specialized server and leverage all the HTTP cache ( for example intermediate proxy )
Al the URL obfuscation technique can be used to protect the content and is announce the support of Playready content protection encryption in order to protect smooth streaming for Live and for VOD.
Smooth streaming enable a new video delivery mechanism that can permit to stream HD content in VOD and Live through Internet with a cool UX .
In next posts we can see in more details the three phase for create and publish content with this technology .