Using Smooth Streaming Media Element for Windows Phone 7


In my previous post I described the avaliability of  Smooth Streaming Media Element (SSME) for Windows Phone (WP) as a part of the Integrated Media Platform vision for IIS Media Services. In this post I would show you how you can use SSME for WP7 in order to reproduce Smooth Streaming (SS) content live and ondemand. In this post I’ll show you how it is possible use directly SSME in a WP project but I would remember that if you are looking for a complete player solution you can leverage the Silverlight Media Framework  that give to you a complete solution based on SSME compatible also with WP .

For first you can download the dll that contain SSME for WP from the latest version of  Smooth Streaming Client SDK . After intalled the SS Client SDK  you can found the Microsoft.Web.Media.SmoothStreaming.dll for WP7    in (your drive):\Program Files (x86)\Microsoft SDKs\IIS Smooth Streaming Client\  select the Windows Phone version.

In order to start with WP you have to download and Install Visual Studio for Windows Phone Tool and Expression Encoder for Windows Phone . You can read all the info for starting with Windows Phone  here.

If you are intrested in a global player that works in every page of your app and in a live smooth streaming audio only, you can read for more info this additional post on this topics.

You can start a new Windows Phone  project from Visual Studio using the specific template project: 

And after that you can add a reference to Microsoft.Web.Media.SmoothStreaming.dll for WP :

In order to insert the SSME in your project you have to add a reference to your MainPage.xaml page, and insert this reference in the <phone:PhoneApplicationPage tag:

 xmlns:SSME=”clr-namespace:Microsoft.Web.Media.SmoothStreaming;assembly=Microsoft.Web.Media.SmoothStreaming”

I suggest to change also the Orientation Support on the page in order to have the landscape orientation that is more useful for wotching videos:

   SupportedOrientations=”Landscape” Orientation=”Landscape”

 this means that you have this PhoneApplicationPage tag in your mainpage.xaml:

 <phone:PhoneApplicationPage
    x:Class=”WindowsPhoneSSMESample.MainPage”
    xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation
    xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml
    xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
    xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
    xmlns:d=”http://schemas.microsoft.com/expression/blend/2008
    xmlns:mc=”http://schemas.openxmlformats.org/markup-compatibility/2006
    xmlns:SSME=”clr-namespace:Microsoft.Web.Media.SmoothStreaming;assembly=Microsoft.Web.Media.SmoothStreaming”
    mc:Ignorable=”d” d:DesignWidth=”728″ d:DesignHeight=”480″
  
    FontFamily=”{StaticResource PhoneFontFamilyNormal}”
    FontSize=”{StaticResource PhoneFontSizeNormal}”
    Foreground=”{StaticResource PhoneForegroundBrush}”
    SupportedOrientations=”Landscape” Orientation=”Landscape”
    shell:SystemTray.IsVisible=”True” Loaded=”PhoneApplicationPage_Loaded”>

 In this way you have referenced a namespace SSME that point to the control that are in Microsoft.Web.Media.SmoothStreaming and you could insert these in your xaml page. Now you can add in ContentPanel Grid this tag in order to insert SSME in your page:

 <!–ContentPanel – place additional content here–>
        <Grid x:Name=”ContentPanel” Grid.Row=”1″ Margin=”12,0,12,0″></Grid>
            <SSME:SmoothStreamingMediaElement x:Name=”video”/>
        </Grid>

We can use  video as a reference from our code to the instance of SSME inserted in the page. From the Loaded Event in MainPage.xaml we can manage the reference and set the SmoothStreamingSource property to the uri of the manifest SS content that we want reproduce:

  video.SmoothStreamingSource = new Uri(“http://localhost/adaptivestreaming/video/vc1/big_buck_bunny_1080p_surround.ism/manifest“); 

You can use Play , Stop, Pause method to control reproduction and you can subscribe all the event in the SSME like      video.CurrentStateChanged  or   video.MediaFailed or   video.ManifestReady  to manage the lifecycle of the SSME in the same way that is possible do for SSME for PC (http://msdn.microsoft.com/en-us/library/ee958035(v=VS.90).aspx) .In order to access to the SSME from the code you have to add to your .cs file this namespace:

using Microsoft.Web.Media.SmoothStreaming;

For example in order to manage Play, Pause, audio volume etc and to show status of Streaming and current Level of bitrates involved in the reproduction we can add these elements to our xaml:

   <!–LayoutRoot is the root grid where all page content is placed–>
    <Grid x:Name=”LayoutRoot” Background=”Transparent”>
     
        <Grid.RowDefinitions>
         
            <RowDefinition Height=”0.90*”></RowDefinition>
           
            <RowDefinition Height=”0.10*”></RowDefinition>
           
        </Grid.RowDefinitions>
      
        <SSME:SmoothStreamingMediaElement x:Name=”video” Grid.Row=”0″ />
       
        <StackPanel Orientation=”Horizontal” Grid.Row=”1″>

           <Button   x:Name=”PlayButton” Width=”50″ Click=”PlayButton_Click” Loaded=”PlayButton_Loaded”/>
           <Button   x:Name=”StopButton” Content=”Stop” Width=”50″ Click=”StopButton_Click” />
           <TextBlock x:Name=”status”/>
           <TextBlock x:Name=”currentBitrate”/>
        
        </StackPanel>
      
    </Grid>

I removed the content grid and added directly in the first layaout grid in our mainpage.xaml:

– two buttons used for Play and Stop directly attached to an handler for click event in the code described before;

– two textblocks used to present status and current bitrates used in reproduction

 

On the code in .cs files I added this handler and additional code in the loaded event:

private void PhoneApplicationPage_Loaded(object sender, RoutedEventArgs e)
        {
            video.CurrentStateChanged += new RoutedEventHandler(video_CurrentStateChanged);
            video.PlaybackTrackChanged +=new EventHandler<TrackChangedEventArgs>(video_PlaybackTrackChanged);
            video.SmoothStreamingSource = new Uri(“http://localhost/adaptivestreaming/CESARONI_DVD02Video.ism/manifest“);  
           
        }

        void video_PlaybackTrackChanged(object sender, TrackChangedEventArgs e)
        {
            currentBitrate.Text = e.NewTrack.Bitrate.ToString();
        }

        void video_CurrentStateChanged(object sender, RoutedEventArgs e)
        {
            status.Text = video.CurrentState.ToString();
        }

        private void StopButton_Click(object sender, RoutedEventArgs e)
        {
            //This should simply stop the playback
            video.Stop();
            //We should also reflect the chang on the play button
            PlayButton.Content = “Play”;
        }

        private void PlayButton_Loaded(object sender, RoutedEventArgs e)
        {
            switch (video.AutoPlay)
            {
                case false:
                    PlayButton.Content = “Play”;
                    break;
                case true:
                    PlayButton.Content = “Pause”;
                    break;
            }
        }

        private void PlayButton_Click(object sender, RoutedEventArgs e)
        {

            //Monitor the state of the content to determine the right action to take on this button being clicked
            //and then change the text to reflect the next action
            switch (video.CurrentState)
            {
                case SmoothStreamingMediaElementState.Playing:
                    video.Pause();
                    PlayButton.Content = “Play”;
                    break;
                case SmoothStreamingMediaElementState.Stopped:
                case SmoothStreamingMediaElementState.Paused:
                    video.Play();
                    PlayButton.Content = “Pause”;
                    break;
            }

        }

 As you can see before I subrcibed in Loaded event two events :

  video.CurrentStateChanged += new RoutedEventHandler(video_CurrentStateChanged);
  video.PlaybackTrackChanged +=new EventHandler<TrackChangedEventArgs>(video_PlaybackTrackChanged);

The first is used to catch change status in SSME and in the handler I write in a specifc TextBlock the current status of SSME, in the second we capture changes in bitrates level used in reproduction and also in this case in the handler I write the current level of bitrate in a specific TextBlock.

The others handlers added to the code are conected to the click event and are used to manage the play stop pause on the SSME. You could read in the code before how this works.

If you would avoid that the application is deactiveted when the Phone go idle or lock (for example you stream audio content and would that the user continue to listen the content ) , you can add to the Loaded Event this code:

PhoneApplicationService.Current.ApplicationIdleDetectionMode = IdleDetectionMode.Disabled;

that prevent deactivation of the app like you can read here and in the specific api description in sdk of wp .

With SSME for WP7 you can play also audio only streaming.  

In order to reproduce SS content with SSME on WP7 you have to pay attention to some additional rules about the type of content supported in the current version of WP7 and SSME 1.5. Here you have a list of supported codecs for SS:  VC1 / H.264 for video, WMA Pro/HEAAC/AACL for audio.

The Maximum Resolution admited on the W7 for H.264 are : 

  • 720 x 480 pixels at 30 fps
  • 720 x 576 pixels at 25 fps

The Current profile for H264 supported on the phone are baseline and main  Level 3.0. 

It is nesessary PAY ATTENTION TO THE FACT THAT IN CURRENT VERSION OF WP IN SOME HW MULTIPLE RESOLUTIONS BITRATE ISN’T SUPPORTED  this means that it is necessary have the same resolution for all the bitrates inlcuded in the range of levels to reproduce in SSME for WP7.  You can check if the device supports multiple resolutions during a play with this api http://msdn.microsoft.com/en-us/library/microsoft.phone.info.mediacapabilities.ismultiresolutionvideosupported(v=VS.92).aspx. In the case that the devices doesnt support multiple resolution and you have a previous encoded content with multiple resolutions or you want create a  single asset for multiple devices , you have to pay attention to this and in the case you can use the StreamInfo.RestrictTracks API   in a ManifestReady event , in order to restrict the range of bitrates used in reproduction . For example imagine to have a manifest that have for biterates <= of 600K the same resolutions and for bitrates >  400k a variable resolutions in order to meet quality requirements of others devices like a PC, in the case tha the  is multipleresolutionvideo supported respond with false you can add the code below in the manifest ready event on WP in order to limit the bitarates used in the reproduction :

 void video_ManifestReady(object sender, EventArgs e)
        {
            if (video == null)
            {
                return;
            }

            foreach (SegmentInfo segment in video.ManifestInfo.Segments)
            {
                foreach (StreamInfo streaminfo in segment.AvailableStreams)
                {
                    if (MediaStreamType.Video == streaminfo.Type)
                    {
                        List<TrackInfo> tracks = new List<TrackInfo>();

                        foreach (TrackInfo track in streaminfo.AvailableTracks)
                        {
                            if (track.Bitrate <= 400000)
                            {
                                tracks.Add(track);
                            }
                        }
                        streaminfo.RestrictTracks(tracks);
                    }
                }
            }
        }

You can subcribe the ManifestReady event in SSME called video in this way :  video.ManifestReady += new EventHandler<EventArgs>(video_ManifestReady);

In alternative you can create an adhoc manifest, server side , one for every devices and point every devices to the right manifest.

Propose an encoding profile is difficult because there are a lot of parameters to consider like bandwith avaliable type of content etc, but on my experience in others project that leverare SS for WP7 I can suggest to have in the manifest of your content this generic profile:

  Use Main Level 3.0  with 640×352 for 16:9 with this levels: 1000 k(Full framerate but <=30) , 800k (Full framerate but <=30) , 600k (Full framerate but <=30) , 400k (Full framerate but <=30) , 200k (half framerate <=15), 100 k(half framerate <=15)

If you would serve the same content to WP7, PC and Apple Devices like iPhone and iPad leveraging the capability of IIS Media Services 4.0 and IIS Transform Manager , you have to built a profile using H264 Baseline Level 3.0 and pay attention to the resolutions in differents bitrates levels and Transmux the SS also in Apple Adaptive Streaming.

If you have to stream a to low level of the maximu 1000k suggested before , you can leverage these profiles for WP7 and iPhone/iPad and in the case add more level fo PC using variable resolution for profiles > 600k and use RestrictTracks API on PC for the minimu bitrates for WP7 for the maximum bitrate and during the transmux operation on IIS Transform Manager and IIS MS 4,0 (in the case of live streaming ) limiting profile fo Apple devices to 600k :

4:3 use Baseline Level 3.0  with 480×360 with this levels: 600 k(Full framerate but <=30) ,  400k (Full framerate but <=30) , 200k (half framerate <=15), 100 k(half framerate <=15)

 16:9 use Baseline Level 3.0  with 480×272 with this levels: 600 k(Full framerate but <=30) ,  400k (Full framerate but <=30) , 200k (half framerate <=15), 100 k(half framerate <=15)

 If you don’t need to cover old iPhone you could move to Main Level 3.0 with the same resolution in order to increase the quality.

Pay attention to the fact that  profiles suggested before are only a sample and not necessary these profile are good for your scenarios. It is necessary test it  and adjust it to your requirements in order to have the best solution.

 

 

Comments (11)

  1. Emanuele says:

    Very good. It's Fantastic.

  2. Vinit says:

    Is there a way to stream only audio using SSME?

    I have tried the following and i get invalidoperation exception . Here Player is an SSME object

    Player.SmoothStreamingPlaybackMode = PlaybackMode.AudioOnly;

    Player.SmoothStreamingSource = null;

    Player.Source = null;

    Player.SmoothStreamingSource = new Uri(http://xxxx/song.mp3);

    Player.Source = new Uri("http://xxxx/song.mp3&quot;);

    Player.AutoPlay =true;

    Player.Play();

    Is there a way this can be achieved?

  3. Vinit says:

    Is there a way to acheive audio only playback mode using SSME?

    I'm running in to invalidoperation exception while trying to do that. Here is my sample code

                       Player.SmoothStreamingPlaybackMode = PlaybackMode.AudioOnly;

                       Player.SmoothStreamingSource = null;

                       Player.Source = null;

                       Player.SmoothStreamingSource = new Uri("http://xxxx/song.mp3&quot;);

                       Player.Source = new Uri("http://xxxx/song.mp3&quot;);

                       Player.Play();

    Any help would be apprecaited.

  4. Vinit, SSME is focalized to Smooth Streaming format. If you would reproduce .mp3 in progressive download  I suggest to use a standard MediaElement and not SSME. This means use <MediaElement Tag in the in the xaml and not the smootstreaming mediaelement. In this case you have to set only Source property . By default Autoplay is true if you set the source Play start automatically. If you would try to use SSME you have to write:                  

                      Player.AutoPlay=false;

                      Player.Source = new Uri("http://xxxx/song.mp3&quot;);

                      Player.Play();

    Where Player is or SSME or MediaElement . But I repeat if you have to deliver only progressive download I suggest to use MediaElement .

    Pay attention to the fact that to test the app on the phone if the phone is connected to the PC you have to use the utility wpconnect.exe to connect the phone to Visual Studio and not Zune because if you use Zune to connect the phone to the PC streaming doesn't work read msdn.microsoft.com/…/gg180729(v=VS.92).aspx and colinizer.com/…/wpdtptconnectwpconnect-tool-in-the-windows-phone-developer-tools-october-2010-update

  5. Vinit says:

    Thanks Giuseppe. I already use mediaelement in my app, but have seen issues where the quality of the streaming is not that great. So, i wanted to try adaptive streaming using SSME and see if there is any difference.

  6. Vinit, in order to use Smooth Streaming it is necessary encode content in this format . You can't use .mp3 , it is necessary produce a Smooth Streaming content. I suggest to you this post blogs.msdn.com/…/move-forward-internet-video-streaming-iis7-media-service-and-smooth-streaming.aspx  and this article to start to learn more about SS learn.iis.net/…/smooth-streaming-primer  

  7. Vinit says:

    Hi Giuseppe,

    Thank you so much for all the information. This is really helpful and i think i'll stick with mediaelement for now.

    I have another question for you. I want to add few mms radio links to my app. The problem is mediaelement doesnt support mms. Smoothstreaming does work if i hardcode the source in the xaml file. If i try to use binding, it doesnt work. it keeps complaining that the source is not set. How do i fix that?

    Thanks in advance.

    -Vinit

  8. Hi , on WP7 with Silverlight MediaElement you could ms-wmsp ove http, it works but it isn't supported . In order to put at work Windows Media Services (wms) live streaming on WP7 MediaElement it is necessary create a pubblishing point that stream  wms ove http. The file extension url of the stream MUST match the encoded source. For example in the case of audio streaming  URL must have .wma extension and protocol prefix MUST have http://. For example you can have : http://servername/pubpointname.wma and you can use from a MediaElement:

                     Player.AutoPlay=false;

                     Player.Source = new Uri("http://servername/pubpointname.wma ");

                     Player.Play();

    and streaming works. It is not supported but works. It is important correct extension that MUST

    match the encoded source and wms ove http.

    I suggest to you to try to use Smooth Streaming using an harware encoder or Expression Encoder 4 Pro. If you have to stream audio only with Smooth Streaming you could do this usinng a campatible encoder and SSME for WP7. You could use also Expression Encoder 4 Pro but current version doesn't suppurt audio anly streaming but you can use a trick: you can select a single profile video with a very low level of bitrate and insert anytype of video signal in input to EE 4 and insert in input the audio that you want stream. In the client side using SSME for WP7 you can select stream audio only and you can ignore the video signal and consume audio only

  9. Shiva says:

    Wonderful post! Is it possible to provide a buffer object as a source in either of MediaElement or SSME? I haven't come across such a scenario. Have you tried it Giuseppe?

  10. Luc says:

    Hello,

    I was experiencing a  AG_E_NETWORK_ERROR and I've replaced the MediaElement by a SSME:SmoothStreamingMediaElement. Now after about 20 sec I get « Too many consecutive chunk parse errors (5) – halting playback ».

    I took the url "http:/…/xxxx/Manifest" and opened it into VLC player on a PC after tiny format changes : "rtsp://…/xxxx" suggested by the provider

    I was able to display the stream.

    Here are the CODEC informations returned by VLC:

    * Codec detected by VLC

    * Stream 0

    *  Type : Audio

    *  Codec : MPEG AAC Audio (mp4a)

    *  Channels : 1

    *  Freq : 48000 Hz

    * Stream 1

    *  Type : Video

    *  Codec : H264 – MPEG-4 AVC (part 10)(h264)

    * Stream 2

    *  Type : Audio

    *  Codec : MPEG AAC Audio (mp4a)

    *  Channels : Stereo

    *  Sampling freq : 48000 Hz

    * Stream 3

    * Type : Video

    *  Codec : H264 – MPEG-4 AVC (part 10)(h264)

    *  Res: 320×240

    *  Pic avg : 59.940059

    I think that this is WP7 compatible CODEC but I'm not really in touch of the video technologies. For example I don't understand why there are 4 streams…

    Anyway do you confirm it's compatible ?

    I'm really appreciate your help.

    Cheers

    luc

  11. Rain says:

    Thanks for your post !