Thoughts On Video on 3D in WPF

  • Article

I have uploaded Sandbox3D into the WPF Gallery.  I added back the X, Y and Z axes using ScreenSpaceLines3D from Daniel Lehenbauer.  Most interestingly, I show two different techniques for mapping video on 3D.  One is using the VideoDrawing with a MediaPlayer, like this:

private void DrawVideoOnMesh(VideoDrawing vd, GeometryModel3D gm3d, Uri uri)
{
MediaPlayer mp = new MediaPlayer();
mp.Open(uri);
mp.MediaEnded += new EventHandler(mp_MediaEnded);
vd.Player = mp;
vd.Rect = new Rect(0, 0, 5, 10);
DrawingBrush db = new DrawingBrush();
db.Drawing = vd;
Brush br = db as Brush;
MaterialGroup mg = new MaterialGroup();
mg.Children.Add(new DiffuseMaterial(br));
gm3d.Material = mg;
vd.Player.Play();
}

The other uses a VisualBrush with a MediaElement:

private void DrawVisualBrushOnMesh(GeometryModel3D gm3d, Uri uri)
{
myMedia.Source = uri;
myMedia.MediaEnded += new RoutedEventHandler(myMedia_MediaEnded);
VisualBrush vb = new VisualBrush(myMedia);
MaterialGroup mg = new MaterialGroup();
mg.Children.Add(new DiffuseMaterial(vb));
gm3d.Material = mg;
myMedia.Play();
}

Because there are two ways to acheive the effect (gee, being able to do something more than one way never happens in WPF), you may be wondering which you should use. The answer is, of course, it depends. 

From a performance perspective, there is very little difference between the two techiques because both techniques will require WPF to create an intermediary render surface to be able to correctly map the video to 3D. (This intermediary step always happens with a VisualBrush even in 2D, whereas it doesn't happen with the VideoDrawing in 2D, which may be worth noting for doing tricky things with video in 2D, but that is another topic.)

From a code manageability perspective, it is worth pointing out that VideoDrawing requires using code, whereas you could create and control (using Storyboards) a VisualBrush, MediaElement and all the necessary 3D gunk entirely in XAML -- in fact, EID does just this. However, there is the slight strangeness that you have to be rendering that MediaElement somewhere, even if it is offscreen or buried in z-space (which is what I did in Sandbox3D).  

One other note: there are two bugs with using video on 3D in Beta2 to be aware of. First, looping is broken, Second, applying a MaterialGroup that has video a second time to the same model doesn't work. However, both of these issues are known and will be resolved for the next CTP.