Fun with AudioVideoPlayback


I’m sure I’m not alone in that I keep a laundry list of personal coding projects I want to work on when I get free time.  Most of them are fairly short and only take a few hours to complete.  We finished work on the June issue of MSDN Magazine yesterday, leaving me with enough time to check off one of the items on my list.


The project involved Managed DirectX as I needed to be able to play a video file on a Form.  Doing this is incredibly straightfoward:


    Video v = Video.FromFile(filepath);
    v.Owner = parentControl;
    v.Play();


Couldn’t be simpler.  Next, I wanted to be able to control the volume for the video, a task that’s also very straightforward:


    Video v = Video.FromFile(filepath);
    v.Owner = parentControl;
    v.Play();
    v.Audio.Volume = -10000; // silence


Piece of cake.  Later on, I stop the video:


    v.Stop();


and to remove the frozen image from the parent control on which the video was playing, I reassign the Video’s owner to be a hidden picture box (thanks to Christopher for the idea):


    PictureBox hiddenBox = new PictureBox();
    hiddenBox.Visible = false;
    …
    v.Owner = hiddenBox;


Again, too easy.  But here’s where I hit a snag.  Microsoft.DirectX.AudioVideoPlayback.Video implements IDisposable which lets you immediately free the unmanaged resources held by the Video object.  I figured I could simply do:


    v.Dispose();


and be done with it.  (insert best buzzer noise here).  Unfortunately, even with this in place, the resources were not being free’d.  I looked at the disassembly for Video and noticed that while Microsoft.DirectX.AudioVideoPlayback.Audio (an instance of which is returned from Video.Audio) implements IDisposable, the Video.Dispose method was not calling Dispose on the instance of Audio that was returned to me in my previous usage of v.Audio.Volume.  It makes sense that both Audio and Video would hold onto unmanaged resources related to the video (in my case, an instance of my MPEG2 decoder was sticking around as was a huge amount of allocated memory), so I modified my code as follows:


    v.Audio.Dispose();
    v.Dispose();


Didn’t help.  Back to ildasm to view Video.get_Audio() (the get accessor for the Video object’s Audio property):


    .method public specialname instance class Microsoft.DirectX.AudioVideoPlayback.Audio
            get_Audio() cil managed
    {
      // Code size       16 (0x10)
      .maxstack  4
      .locals (class Microsoft.DirectX.AudioVideoPlayback.Audio V_0)
      .try
      {
        IL_0000:  ldarg.0
        IL_0001:  newobj    
         
instance void Microsoft.DirectX.AudioVideoPlayback.Audio::.ctor(
          
class Microsoft.DirectX.AudioVideoPlayback.Video)
        IL_0006:  stloc.0

        IL_0007:  leave.s    IL_000e
      }  // end .try
      catch [mscorlib]System.Exception
      {
        IL_0009:  pop
        IL_000a:  ldnull
        IL_000b:  stloc.0
        IL_000c:  leave.s    IL_000e
      }  // end handler
      IL_000e:  ldloc.0
      IL_000f:  ret

    } // end of method Video::get_Audio


This decompiles to C# something like the following (although I believe the original code was written in MC++):


    public Audio Audio
    {
        get
        {
            Audio tempAudio;
            try
            {
                tempAudio = new Audio(this);
            }
            catch(Exception)
            {
                tempAudio = null;
            }
            return tempAudio;
        }
    }


I’ve highlighted in red the important lines.  No wonder calling v.Audio.Dispose() wasn’t helping… in fact, it could very well have been making things worse.  Every call to v.Audio returns a new instance of Audio, so I was disposing a brand new object, not the one I originally set the volume on.  Ugh.  I had been expecting something more along the lines of:


    private Audio _audio;
    …
    public Audio Audio
    {
        get
        {
            if (_audio == null) _audio = new Audio(this);
            return _audio;
        }
    }


where the Audio property would always return the same instance of Audio (assuming the underlying video hadn’t changed) instead of creating a new one each time.  The fix? I simply changed my code to not only store a reference to the Video object v, but also to the initial result of v.Audio.  I then only use v.Audio when changing the volume in the future (which I do a few times).  When it’s time to clean things up, instead of:


    v.Audio.Dispose();
    v.Dispose();


I simply do:


    Audio a = v.Audio;
    … do whatever I need to do with a instead of v.Audio…
    a.Dispose();
    v.Dispose();


And with that, the memory usage of my application returned to normal and the unnecessary instances of my MPEG2 decoder were released and went away.

Comments (23)

  1. David M. Kean says:

    The code:

    public Audio Audio

    {

    get

    {

    Audio tempAudio;

    try

    {

    tempAudio = new Audio(this);

    }

    catch(Exception)

    {

    tempAudio = null;

    }

    return tempAudio;

    }

    }

    breaks the design rule that a property shouldn’t return a new instance of a class. Sound like the developers weren’t following .NET guidelines…

  2. Adrian says:

    Hi there,

    Thanks for an excellent article. Granted it’s resonably simple, but it’s exactly what I’m looking for at the moment, and have not found many other good resources with examples to play video files, those that did complained about bugs with memory allocation that you seemed to have nailed down.

    In particular though, I liked the idea of browsing the MSIL code to see what’s happening. Having codes in assember many years ago, I should be able to find my way around, so it’s an excellent tip, which I don’t think I would have though of straight away.

    Thanks, Adrian.

  3. Brian Weeres says:

    How did you determine that resources were not being freed up?

  4. Stephen Toub says:

    1) Even after running the GC, waiting for pending finalizers, and running the GC again, the hundreds of megabytes of memory consumed by my application never decreased. After loading a few videos, playing them, and "releasing" them, my process was still consuming close to half a gig of virtual memory.

    2) My MPEG2 decoder is configured to display an icon in the systray when active, and each instance of the decoder displays its own icon. So, for example, after playing three videos, three decoder icons still remained.

  5. Achim Thurner (athurner@bigfoot.com.no_spam says:

    Hi Stephan

    Thanks for your article. In your code sample you change the volume of the video I’m running into following problem using the audio object: The video events are not fired anymore after accessing the video.audio object:

    oVideo = new Video(ofdOpen.FileName);

    // Adding the event handler

    oVideo.Ending += new System.EventHandler(this.ClipEnded);

    oVideo.Starting += new System.EventHandler(this.ClipStarting);

    oVideo.Pausing += new System.EventHandler(this.ClipPausing);

    oVideo.Stopping += new System.EventHandler(this.ClipStopping);

    oVideo.Owner = this;

    // Start playing now

    oVideo.Play();

    This works fine and the events are fired correctly.

    But after accessing the audio object this way

    MessageBox.Show(oVideo.Audio.Volume.ToString() );

    (the result is 0 and not correct because a new instance of Audio was called)

    or after accessing the audio object the way:

    oAudio = oVideo.Audio;

    MessageBox.Show(oAudio.Volume.ToString() );

    the events are not fired anymore.

    Do you know why?

  6. Bastian Buch says:

    Hi Achim… I had the same problems… Maybe u could find an answer here:

    http://www.codeproject.com/cs/media/DirectX9_media_playback.asp

  7. Pir Qaiser says:

    Hi!

    It was a good discussion. Could you let me know how to merge two video files or merge an image to the end of video?

    Thanks.

  8. Balogh Attila says:

    Hy there. This i a great article about AudioVideoPlayback. When i read whats the problem i literaly wanted to bang my head against the wall. And there are other problems with the Video class. Video.RenderToTexture() ruins your original rendering loop.

    Achim i think i have a solution on the event fireing problem

    after a=v.Audio

    you should sent the ending event the same as the v.Ending event. I did it like this.

    aud = m_video.Audio;

    aud.Volume = GeneralVolume;

    aud.Ending += new EventHandler(m_video_Ending);

    m_video.Owner = owner;

    m_video.Ending += new EventHandler(m_video_Ending);

    And the events fire.

  9. Balogh Attila says:

    Oh sh..! After 70000 consecutive video loads my app used 200mb-s more than normaly. Even if we workaround the audio propert we can’t avoid a minor memory leak. How come Microsoft won’t recompile this dll with the correct code??? They would save us from alot of headache.

  10. mohamad says:

    how can i insert a "audio level meter" by using managed Direct x ?

  11. L.S. Jensen says:

    Thanks.

    This also solved my problem where the video-file was locked (could not be deleted) until the application was terminated.

  12. Jaycee says:

    Hi

    I’m French, so sorry for my english. I’ve try to do the same, but it doesn’t work. Can you send by email the dll at hager.jc@gmail.com ?

    Thanks

  13. fazz says:

    Nice article but it doesnt work on my project.

    Im working with Vb.net and i do

    If (_video IsNot Nothing) Then

               _video.Stop()

               Dim a As Audio = _video.Audio

               a.Dispose()

               a = Nothing

               Dim v As Video = _video

               v.Dispose()

               v = Nothing

           End If

    but memory is still allocated : FFDSHOW  tray icons dont disapear (neither audio nor video) and memory usage don’t decrease

    Thank for your help

  14. maliodtipke@gmail.com says:

    don’t know is it fits here but calling dispose is totaly wrong becouse GC do that automaticly.

    if you look for amount of used memory first look for allocated memory then used memory. it’s big diference

  15. Haris Jamil Dar says:

    i have done the above mentioned method which was  Audio a = v.Audio;

       … do whatever I need to do with a instead of v.Audio…

       a.Dispose();

       v.Dispose();

    but the thing is in C# i have to instantiate the audio object before using it so i have to give the path of the movie file to its constructor. i have tried the possible things u have mentioned in the article but the problem of much memory used is not getting solved. as soon as one movie file gets finished and other starts the performance goes down and the movie gets really slow. any other solution to this problem.

  16. I dont really understand the function Volume… It seems it works somewhere between -10000 and – 500? can someone tell me the exact range of that funtion?

    (im sorry for that bad english and noob question)

    # Regards,

    # Tom

  17. Max says:

    Do you have an idea why the .ending and .pausing events dont fire? Simple example:

    using System;

    using Microsoft.DirectX.AudioVideoPlayback;

    namespace Test

    {

       class Program

       {

           static Audio audioObject;

           static void Main(string[] args)

           {

               audioObject = new Audio(@"C:test.mp3");

               audioObject.Starting += new EventHandler(audioObject_Pausing);

               audioObject.Play();

               System.Threading.Thread.Sleep(1000);

               audioObject.Pause();

               System.Threading.Thread.Sleep(1000);

               audioObject.Play();

           }

           static void audioObject_Pausing(object sender, EventArgs args)

           {

               Console.WriteLine("Pause Event!");

           }

       }

    }

  18. Josh Stodola says:

    Thanks for posting this.  Unfortunately I did the same digging you did and discovered the same problem.  And THEN I decided to ask Google.  Sigh.  One of these days I’ll learn 😉

    BTW, this is one of my favorite namespaces!

  19. Mario.. says:

    Why can’t i use:

    using Microsoft.DirectX.AudioVideoPlayback; ?

    I get an red line under DirectX..

  20. KWS says:

    Thanks for your posting!

    It’s clear my terrible problem.

  21. PingPl says:

    Thanks for this article, it was really helpful :).

    Now, some anwsers to the other comments.

    Events of the video object aren’t fired if you set video’s volume using Audio.Volume property. Often it’s needed to use it though, so you have to create solutions that don’t use those events.

    From what I’ve found the range of the Volume is [-10000; 0]. However, I used -6000 as a lower bound and it’s still silence on my machine. Let me know if you hear something with Volume set to -6000.

    "…

    in C# i have to instantiate the audio object before using it so i have to give the path of the movie file to its constructor. i have tried the possible things u have mentioned in the article but the problem of much memory used is not getting solved. as soon as one movie file gets finished and other starts the performance goes down and the movie gets really slow.

    …"

    As to the issue with c# I’m not sure what you are talking about. You should try creating new video object each time the new video is loaded. When you create Video, the Audio object associated with it will be initialized as well. I used the described solution in C# application and after loading several videos the memory usage doesn’t increase noticeably (it was before applying th fix described).

    "…

    Why can’t i use:

    using Microsoft.DirectX.AudioVideoPlayback; ?

    I get an red line under DirectX..

    …"

    Make sure that you’ve added a reference to Microsoft.DrectX.AudioVideoPlayback in the project. To add a reference you have to right-click on the project in the Solution Explorer, choose Add reference… and choose the reference you want to add from the list (.NET tab). You also may need to download DirectX SDK to make it work.

  22. docwho says:

    i experienced this problem when i set the audio.volume of the video, if you don’t touch the audio.volume the video dispose works fine…