How to await a MediaElement (PlaySound in Windows 8)
[This post is part of a series How to await a storyboard, and other things]
Let’s look at making MediaElement awaitable. This is the kind of idiom I’d like to use:
Try
Await mediaElement1.OpenAsync(New Uri("ms-appx:///Assets/boooo.mp3"))
Await mediaElement1.PlayAsync()
Catch ex As Exception
...
End Try
Q. Why do I want to await it? Why not just set its AutoPlay property to True, then set its Source property, and let it go?
A. Well, AutoPlay is fine for simple scenarios. But it doesn’t compose in the nice way that await does. It doesn’t let me use my familiar exception handling blocks - inside I’d have to sign up to a MediaFailed event handler. And it doesn’t let me compose in other interesting ways, e.g.
Await Task.WhenAll(storyboard1.PlayAsync(), mediaElement1.Play())
ALERT. MediaElement must be in the visual tree. Typically that means you create it as a child of your current Xaml page. If you want it to play globally, i.e. independent of which page is being shown, read this thread: https://social.msdn.microsoft.com/Forums/en-US/winappswithcsharp/thread/241ba3b4-3e2a-4f9b-a704-87c7b1be7988/
Jumping straight to the answer, here’s my attempt at code that supports awaitability. There may be bugs in it: it’s complex, and this is my first attempt. Requirements: neither of these operations is re-entrant: i.e. if other operations are performed on a given media element while either one is in progress, it might go awry.
<Extension> Function OpenAsync(media As MediaElement, uri As Uri,
Optional cancel As CancellationToken = Nothing) As Task
Dim tcs As New TaskCompletionSource(Of Object)
If cancel.IsCancellationRequested Then
tcs.SetCanceled()
Return tcs.Task
End If
If media.CurrentState = MediaElementState.Buffering OrElse
media.CurrentState = MediaElementState.Opening OrElse
media.CurrentState = MediaElementState.Playing Then
tcs.SetException(New Exception("MediaElement not ready to open"))
Return tcs.Task
End If
Dim lambdaOpened As RoutedEventHandler = Nothing
Dim lambdaChanged As RoutedEventHandler = Nothing
Dim lambdaFailed As ExceptionRoutedEventHandler = Nothing
Dim cancelReg As CancellationTokenRegistration? = Nothing
Dim removeLambdas As Action = Sub()
RemoveHandler media.MediaOpened, lambdaOpened
RemoveHandler media.MediaFailed, lambdaFailed
RemoveHandler media.CurrentStateChanged, lambdaChanged
If cancelReg.HasValue Then cancelReg.Value.Dispose()
removeLambdas = Sub() Return
' in case two lambdas get fired one after the other
End Sub
lambdaOpened = Sub()
removeLambdas()
tcs.TrySetResult(Nothing)
End Sub
lambdaFailed = Sub(s, e)
removeLambdas()
tcs.TrySetException(New Exception(e.ErrorMessage))
End Sub
lambdaChanged = Sub()
If media.CurrentState <> MediaElementState.Closed Then Return
removeLambdas()
tcs.TrySetCanceled()
End Sub
AddHandler media.MediaOpened, lambdaOpened
AddHandler media.MediaFailed, lambdaFailed
AddHandler media.CurrentStateChanged, lambdaChanged
media.Source = uri
If Not tcs.Task.IsCompleted Then
' The above condition guards against lambas being invoked by Source assignment
cancelReg = cancel.Register(
Sub()
Dim dummy = media.Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal,
Sub()
media.ClearValue(MediaElement.SourceProperty)
End Sub)
End Sub)
End If
Return tcs.Task
End Function
<Extension> Function PlayAsync(media As MediaElement,
Optional cancel As CancellationToken = Nothing) As Task
Dim tcs As New TaskCompletionSource(Of Object)
If cancel.IsCancellationRequested Then
tcs.SetCanceled()
Return tcs.Task
End If
If media.CurrentState <> MediaElementState.Paused Then
tcs.SetException(New Exception("MediaElement not ready to play"))
Return tcs.Task
End If
Dim lambdaEnded As RoutedEventHandler = Nothing
Dim lambdaChanged As RoutedEventHandler = Nothing
Dim cancelReg As CancellationTokenRegistration? = Nothing
Dim removeLambdas As Action = Sub()
RemoveHandler media.MediaEnded, lambdaEnded
RemoveHandler media.CurrentStateChanged, lambdaChanged
If cancelReg.HasValue Then cancelReg.Value.Dispose()
removeLambdas = Sub() Return
End Sub
lambdaEnded = Sub()
removeLambdas()
tcs.TrySetResult(Nothing)
End Sub
lambdaChanged = Sub()
If media.CurrentState <> MediaElementState.Stopped Then Return
removeLambdas()
tcs.TrySetCanceled()
End Sub
AddHandler media.MediaEnded, lambdaEnded
AddHandler media.CurrentStateChanged, lambdaChanged
media.Play()
If Not tcs.Task.IsCompleted Then
cancelReg = cancel.Register(
Sub()
Dim dummy = media.Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal,
Sub()
media.Stop()
End Sub)
End Sub)
End If
Return tcs.Task
End Function
What is the interactive behavior of MediaElement?
There’s no way I could make MediaElement awaitable without knowing the details of how it behaves. I did my PhD on the “pi calculus”, a sort of mathematical algebra for describing and specifying interactive behaviors. Now I’m not going to delve into mathematics here, but I believe that software engineering requires the same careful approach when you get down to the edge cases of async. Turning a complex event-based thing into a Task-based thing is one of those edge cases.
When studying the interactive behavior of a thing, then you HAVE to understand its complete state diagram. You have to understand in which states you’re allowed to trigger which stimuli (e.g. invoking the Play() method), and what their effect will be. You have to understand in which states the thing might have its own internal stimuli (e.g. reaching the end of a piece of media) and what effects these will have. If the documentation fails to specify all this, then you'll have to discover it by yourself.
I’m not a XAML expert. I don’t know the full details of how MediaElement behaves. But from experiment, this is what I believe is its state diagram:
State |
Possible internal transitions |
Upon setting Source |
Upon reset of Source |
Upon invoking Play() |
Upon invoking Stop() |
Closed |
|
-> Opening |
|
|
|
Opening |
-> Paused (MediaOpened) -> Closed (MediaFailed) |
? |
-> Closed |
|
|
Paused |
|
-> Opening |
-> Closed |
-> Buffering -> Playing |
-> Stopped |
Buffering |
-> Playing |
? |
? |
|
-> Stopped |
Playing |
-> Buffering -> Paused (MediaEnded) |
? |
? |
|
-> Stopped |
Stopped |
|
-> Opening |
? |
-> Buffering -> Playing |
|
Read the table like this: “If the media element is currently in <State>, then it might transition “->” to another state either due to its internal workings or due to some method being invoked. If it does transition, then first its CurrentState property is changed, then it may fire a specified event on the UI thread, then it fires the CurrentStateChanged event on the UI thread.”
The media element starts in state “Closed” upon construction. If its Source property is set in XAML, then that has the same effect as setting the source property in code. The column for “reset of source” refers to invoking the method “mediaElement1.ClearValue(MediaElement.SourceProperty)”. That is the only way I have found to completely reset the Source property from code.
I noticed that the transition “Opening -> Closed (MediaFailed)” can take up to 10 seconds in the case of an https:// uri which either doesn’t exist (returns an error http code) or where your app lacks Internet Client permissions.
I haven’t tried to characterize every possible method than can be invoked on mediaElement. I only characterized the ones needed to support my goals.
Conclusions
Does this code look way more complicated to you than it should? It does to me. To be clear, the use of the code is very simple
Await mediaElement1.OpenAsync(New Uri("ms-appx:///Assets/boooo.mp3"))
Await mediaElement1.PlayAsync()
It’s just the implementation of awaitability-support that’s complex. Why so complex? Actually, my initial attempt was much simpler. It merely played a media element which already had its Source property set to the right value, and had already opened it successfully.
<Extension> Function PlayAsync1(media As MediaElement) As Task
Dim tcs As New TaskCompletionSource(Of Object)
Dim lambda As RoutedEventHandler = Sub()
RemoveHandler media.MediaEnded, lambda
tcs.SetResult(Nothing)
End Sub
AddHandler media.MediaEnded, lambda
media.Play()
Return tcs.Task
End Function
It got more complex because (0) I wanted to support setting the source property, (1) I wanted to support cancellation, (2) I wanted to figure out the complete state diagram, to know in which states it’s valid to call PlayAsync, and (3) I wanted to know whether PlayAsync might ever end in failure. People often complain that “the Play() method doesn’t do anything”, and the answer is that Play() is only valid after MediaOpened has fired.
My analysis isn’t complete. I don’t know what happens if you set the source at the wrong time. What would happen if you did “Dim t1 = media1.OpenAsync(uri1), t2 = media1.OpenAsync(uri2)” ? Would it end up with a random one of the two URIs opened? Or would it end up in some broken internal state? Or what happens if, in response to a button-click, you did “Await media1.OpenAsync(uri)”, and the user impatiently clicks the button a second time while it was still opening the first one? Would it finish the first open operation? or abort it and start the second open operation? Or would it end in some broken state?
The ultimate problem is that MediaElement has two very different uses – either as a full-blown media playing control with start/stop/seek controls, or just for doing PlaySoundAsync(). The first use isn’t much like a task at all. That’s because there’s no single obvious completion event, and because it isn’t monotonic. A monotonic state diagram is one where you never return to an earlier state. Readonly variables are also monotonic, since they transition from “unassigned” to “assigned-with-value”. Monotonic things are much easier to program against. Compare the state-diagram for MediaElement above, with the simpler state diagram for Tasks that come out of async methods. It’s easy to know where you are with tasks.
State |
Possible internal transitions |
WaitingForActivation |
-> RanToCompletion -> Canceled -> Faulted |
RanToCompletion |
|
Canceled |
|
Faulted |
|
I do wonder whether it might be best to abandon awaitability of MediaElement itself, and encapsulate it into a simpler API like PlaySoundAsync. But this then would have its own problems: how do I pause it? how do I declare it in XAML? how do I open it just once but then play it multiple times? What forms of uri does it support? How to set the volume? How does it find the visual-tree? My encapsulation might end up just as complicated as MediaElement!
Function PlaySoundAsync(uri As Uri, Optional cancel As CancellationToken) As Task
...
End Function