If you drag one clip to the timeline within Windows Movie Maker and preview it you may think that the process that goes on is very similar to when you preview the clip within the collection view. Well, although there are a few similarities there are a lot of differences. When we preview the clip from the collection we render the file directly using DirectShow. When we preview the timeline we use the DirectShow Editing Services (this was codenamed Dexter as you can see if look at the name of its type library, and this is still the way we refer to it internally).
Above is the graph that gets generated when we preview a file from the collection (the Windows Movie Maker sample file in this case). As you can see it’s fairly straightforward. The source filter on the left reads the bits from disc and outputs the raw video and audio streams. You can then see two DMO’s (DirectX Media Objects) that are responsible for decoding those bits into the actual video and audio data. The audio data is then passed to the DirectSound Device filter which will send the bits to your audio card, and the video data is sent to the VMR (Video Mixing Renderer) which will send the video frames to your graphics card.
OK. Now let's see what the graph looks like if you play a clip using DirectShow Editing Services.
This time you’ll see two source filters for the file as Dexter doesn’t share source filters across the video and audio streams. Note that if the same source file is used multiple times within the same timeline Dexter will share the same source filter within the same stream as long as it doesn’t overlap with itself e.g. if you have a transition.
On the left side you’ll also notice another source filter which is used to generate a solid colour, black in this case. Dexter uses this filter whenever there is a gap in video on the timeline, although this shouldn’t actually happen with the way Movie Maker uses it.
Dexter expect all audio within a single stream to be at the same format so you’ll see that after the audio is decoded there are a couple of other filters that convert and repackage the audio to Dexter’s liking. Similarly Dexter expects all the video within one stream to be at a single frame rate so it supplies a Frame Rate Converter filter to do the conversion if necessary. Movie Maker 2 tells Dexter to use a frame rate of 15 frames per second for NTSC and 12.5 for PAL. We use half the normal frame rate so that less data has to be processed through the timeline so we can get better performance. If the supplied video isn’t 320x240 then an extra filter will be inserted before the Frame Rate Converter that will resize the video to 320x240 as this is the resolution we use for preview. (See my blog about preview for further details on this).
After these, come two filters that are essentially the heart of Dexter and they are fundamentally big switches. They have multiple pins coming through and one main pin coming out. In this case we only have one clip so there aren't that many inputs connected. However if you imagine a timeline with a lot of non-contiguous clips then these switches will have lots of connected input pins. As the timeline is played Dexter will output the data from the pin connected to the relevant source at that time on the timeline and it will switch which pin is supplying the data as the timeline progresses as it moves from clip to clip.
After the switches, the audio data is sent to Dexter’s audio renderer and the video data is again sent to the VMR (after going through a color space converter which convert the video to match your graphics card if necessary).
So as you can see, even in this simple case there are quite a few differences between collection playback and timeline playback and you can probably just begin to imagine what some of these graphs can look like on a fairly complicated timeline.