Rendering with XNA Framework 4.0 inside of a WPF application

First some prefaces:

1. This isn’t any official solution to embedding XNA Framework graphics inside a WPF application. This is simply my personal attempt at making it happen.
2. The code here might not be the most optimized, but it certainly gets the job done.

Alright, so yesterday I decided I wanted to start building some tools for games. I had nothing in particular at the time, but knew I wanted to use WPF for the application and XNA Framework for the graphics. Since HiDef is now supported in the XNA Game Studio 4.0 Beta, I decided to go ahead and use Visual Studio 2010 with .NET 4.0 and XNA Framework 4.0. Then the hard part came of how to make WPF and XNA Framework play nice together.

After some searching the interwebs, I found a few ways people had accomplished this:

1. Use the regular WinForm approach and use the WindowsFormsHost inside of WPF. This works but I thought it a bit ugly.
2. Render a frameless window over any panels that were XNA Framework content. This works but also is quite ugly. You’re essentially going to be running an application per view and just hiding the fact that it’s a separate application from the user. Not very nice.
3. Use the D3DImage class to display the content. This seems like the ideal, but since the XNA Framework does not directly expose the native IDirect3D9Surface pointer, the only way to make this work is with a bunch of reflection hackery.

So I decided to do it my own way. My first idea was from looking at the D3DImage approach. I wasn’t going to use the D3DImage, but I did start to think about how to create my own ImageSource such that I could use the WPF Image class to display the content. After heading down the MSDN documentation road for the ImageSource and trying to implement it myself, I quickly found it near impossible to decipher exactly how the Image queried the ImageSource for the pixel data.

I then decided that I wouldn’t even implement my own ImageSource, but rather I’d just stick data into an existing ImageSource, the WriteableBitmap to be precise. So I made a class to wrap up a RenderTarget2D along with a WriteableBitmap and handle transferring data from one to the other:

 /// <summary>
/// A wrapper for a RenderTarget2D and WriteableBitmap 
/// that handles taking the XNA rendering and moving it 
/// into the WriteableBitmap which is consumed as the
/// ImageSource for an Image control.
/// </summary>
public class XnaImageSource : IDisposable
{
    // the render target we draw to
    private RenderTarget2D renderTarget;

    // a WriteableBitmap we copy the pixels into for 
    // display into the Image
    private WriteableBitmap writeableBitmap;

    // a buffer array that gets the data from the render target
    private byte[] buffer;

    /// <summary>
    /// Gets the render target used for this image source.
    /// </summary>
    public RenderTarget2D RenderTarget
    {
        get { return renderTarget; }
    }

    /// <summary>
    /// Gets the underlying WriteableBitmap that can 
    /// be bound as an ImageSource.
    /// </summary>
    public WriteableBitmap WriteableBitmap
    {
        get { return writeableBitmap; }
    }

    /// <summary>
    /// Creates a new XnaImageSource.
    /// </summary>
    /// <param name="graphics">The GraphicsDevice to use.</param>
    /// <param name="width">The width of the image source.</param>
    /// <param name="height">The height of the image source.</param>
    public XnaImageSource(GraphicsDevice graphics, int width, int height)
    {
        // create the render target and buffer to hold the data
        renderTarget = new RenderTarget2D(
            graphics, width, height, false,
            SurfaceFormat.Color,
            DepthFormat.Depth24Stencil8);
        buffer = new byte[width * height * 4];
        writeableBitmap = new WriteableBitmap(
            width, height, 96, 96,
            PixelFormats.Bgra32, null);
    }

    ~XnaImageSource()
    {
        Dispose(false);
    }

    public void Dispose()
    {
        Dispose(true);
    }

    protected virtual void Dispose(bool disposing)
    {
        renderTarget.Dispose();

        if (disposing)
            GC.SuppressFinalize(this);
    }

    /// <summary>
    /// Commits the render target data into our underlying bitmap source.
    /// </summary>
    public void Commit()
    {
        // get the data from the render target
        renderTarget.GetData(buffer);

        // because the only 32 bit pixel format for WPF is 
        // BGRA but XNA is all RGBA, we have to swap the R 
        // and B bytes for each pixel
        for (int i = 0; i < buffer.Length - 2; i += 4)
        {
            byte r = buffer[i];
            buffer[i] = buffer[i + 2];
            buffer[i + 2] = r;
        }

        // write our pixels into the bitmap source
        writeableBitmap.Lock();
        Marshal.Copy(buffer, 0, writeableBitmap.BackBuffer, buffer.Length);
        writeableBitmap.AddDirtyRect(
            new Int32Rect(0, 0, renderTarget.Width, renderTarget.Height));
        writeableBitmap.Unlock();
    }
}

With that in place, I now had a mechanism for rendering XNA Framework content (into the RenderTarget2D) and moving that into the WriteableBitmap which could be consumed by WPF.

Next I set out to make a custom XnaControl that I could add to my window. The XAML for the XnaControl is very minimal given that it just wraps an Image:

 <UserControl x:Class="MyApp.XnaControl"
        xmlns="https://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml"
        MinHeight="50" MinWidth="50" Background="CornflowerBlue">
    <Image x:Name="rootImage" />
</UserControl>

Nothing special there. I just set it up with a minimum size of 50x50 because that seemed good to me and of course used a CornflowerBlue background so that I can see the control more easily in design view. Then I just have a basic Image which will get hooked up in code.

Next I wrote up the codebehind for my XnaControl. The primary responsibility of the codebehind is to handle setting up the GraphicsDevice along with our XnaImageSource and managing that during rendering. The first bit I needed was to borrow the GraphicsDeviceService.cs from the WinForms Sample and modify it a bit to my liking. Namely since I am using one RenderTarget2D per view, the backbuffer size doesn’t matter so I can remove some parameters and methods for that. I also had to update it to work with the new APIs in XNA Framework 4.0 and eventually ended up with this:

 /// <summary>
/// Helper class responsible for creating and managing the GraphicsDevice.
/// All GraphicsDeviceControl instances share the same GraphicsDeviceService,
/// so even though there can be many controls, there will only ever be a 
/// single underlying GraphicsDevice. This implements the standard 
/// IGraphicsDeviceService interface, which provides notification events for 
/// when the device is reset or disposed.
/// </summary>
public class GraphicsDeviceService : IGraphicsDeviceService
{
    // Singleton device service instance.
    private static GraphicsDeviceService singletonInstance;

    // Keep track of how many controls are sharing the singletonInstance.
    private static int referenceCount;

    /// <summary>
    /// Gets the single instance of the service class for the application.
    /// </summary>
    public static GraphicsDeviceService Instance
    {
        get
        {
            if (singletonInstance == null)
                singletonInstance = new GraphicsDeviceService();
            return singletonInstance;
        }
    }

    // Store the current device settings.
    private PresentationParameters parameters;

    /// <summary>
    /// Gets the current graphics device.
    /// </summary>
    public GraphicsDevice GraphicsDevice { get; private set; }

    // IGraphicsDeviceService events.
    public event EventHandler<EventArgs> DeviceCreated;
    public event EventHandler<EventArgs> DeviceDisposing;
    public event EventHandler<EventArgs> DeviceReset;
    public event EventHandler<EventArgs> DeviceResetting;

    /// <summary>
    /// Constructor is private, because this is a singleton class:
    /// client controls should use the public AddRef method instead.
    /// </summary>
    GraphicsDeviceService() { }

    /// <summary>
    /// Creates the GraphicsDevice for the service.
    /// </summary>
    private void CreateDevice(IntPtr windowHandle)
    {
        parameters = new PresentationParameters();

        // since we're using render targets anyway, the 
        // backbuffer size is somewhat irrelevant
        parameters.BackBufferWidth = 480;
        parameters.BackBufferHeight = 320;
        parameters.BackBufferFormat = SurfaceFormat.Color;
        parameters.DeviceWindowHandle = windowHandle;
        parameters.DepthStencilFormat = DepthFormat.Depth24Stencil8;
        parameters.IsFullScreen = false;

        GraphicsDevice = new GraphicsDevice(
            GraphicsAdapter.DefaultAdapter, 
            GraphicsProfile.HiDef, 
            parameters);

        if (DeviceCreated != null)
            DeviceCreated(this, EventArgs.Empty);
    }

    /// <summary>
    /// Gets a reference to the singleton instance.
    /// </summary>
    public static GraphicsDeviceService AddRef(IntPtr windowHandle)
    {
        // Increment the "how many controls sharing the device" 
        // reference count.
        if (Interlocked.Increment(ref referenceCount) == 1)
        {
            // If this is the first control to start using the
            // device, we must create the device.
            Instance.CreateDevice(windowHandle);
        }

        return singletonInstance;
    }

    /// <summary>
    /// Releases a reference to the singleton instance.
    /// </summary>
    public void Release()
    {
        // Decrement the "how many controls sharing the device" 
        // reference count.
        if (Interlocked.Decrement(ref referenceCount) == 0)
        {
            // If this is the last control to finish using the
            // device, we should dispose the singleton instance.
            if (DeviceDisposing != null)
                DeviceDisposing(this, EventArgs.Empty);

            GraphicsDevice.Dispose();

            GraphicsDevice = null;
        }
    }
}

If you’ve seen the WinForm version, you’ll see it’s fairly similar with a few things removed here and there. I also moved the device creation to a CreateDevice method instead of the constructor and made a public property to get the instance. This seemed useful to me so external classes can hook the DeviceCreated event (and others) so they can load and manage content.

Now that we’re done there, we can write the XnaControl codebehind which will get us rendering our content into the Image control.

 public partial class XnaControl : UserControl
{
    private GraphicsDeviceService graphicsService;
    private XnaImageSource imageSource;

    /// <summary>
    /// Gets the GraphicsDevice behind the control.
    /// </summary>
    public GraphicsDevice GraphicsDevice
    {
        get { return graphicsService.GraphicsDevice; }
    }

    /// <summary>
    /// Invoked when the XnaControl needs to be redrawn.
    /// </summary>
    public Action<GraphicsDevice> DrawFunction;

    public XnaControl()
    {
        InitializeComponent();

        // hook up an event to fire when the control has finished loading
        Loaded += new RoutedEventHandler(XnaControl_Loaded);
    }

    ~XnaControl()
    {
        imageSource.Dispose();

        // release on finalizer to clean up the graphics device
        if (graphicsService != null)
            graphicsService.Release();
    }

    void XnaControl_Loaded(object sender, RoutedEventArgs e)
    {
        // if we're not in design mode, initialize the graphics device
        if (DesignerProperties.GetIsInDesignMode(this) == false)
        {
            InitializeGraphicsDevice();
        }
    }

    protected override void OnRenderSizeChanged(SizeChangedInfo sizeInfo)
    {
        // if we're not in design mode, recreate the 
        // image source for the new size
        if (DesignerProperties.GetIsInDesignMode(this) == false && 
            graphicsService != null)
        {
            // recreate the image source
            imageSource.Dispose();
            imageSource = new XnaImageSource(
                GraphicsDevice, (int)ActualWidth, (int)ActualHeight);
            rootImage.Source = imageSource.WriteableBitmap;
        }

        base.OnRenderSizeChanged(sizeInfo);
    }

    private void InitializeGraphicsDevice()
    {
        if (graphicsService == null)
        {
            // add a reference to the graphics device
            graphicsService = GraphicsDeviceService.AddRef(
                (PresentationSource.FromVisual(this) as HwndSource).Handle);

            // create the image source
            imageSource = new XnaImageSource(
                GraphicsDevice, (int)ActualWidth, (int)ActualHeight);
            rootImage.Source = imageSource.WriteableBitmap;

            // hook the rendering event
            CompositionTarget.Rendering += CompositionTarget_Rendering;
        }
    }

    /// <summary>
    /// Draws the control and allows subclasses to override 
    /// the default behavior of delegating the rendering.
    /// </summary>
    protected virtual void Render()
    {
        // invoke the draw delegate so someone will draw something pretty
        if (DrawFunction != null)
            DrawFunction(GraphicsDevice);
    }

    void CompositionTarget_Rendering(object sender, EventArgs e)
    {
        // set the image source render target
        GraphicsDevice.SetRenderTarget(imageSource.RenderTarget);

        // allow the control to draw
        Render();

        // unset the render target
        GraphicsDevice.SetRenderTarget(null);

        // commit the changes to the image source
        imageSource.Commit();
    }
}

Again, nothing in here too crazy if you read the comments. Two notes in here about performance:

1. Anytime the control is resized, we recreate the image source. We do this because we at least need to fix up the WriteableBitmap and at this point I wasn’t too worried about the performance cost of disposing and recreating the render target.
2. We draw everytime WPF’s Rendering event fires which is 60fps. This is a bit overkill for most scenarios where you only care about drawing when something has changed. It might be worth investigating a way to allow both scenarios with the use of a flag.

Now that we’re all set up, we can actually use the control in a window. Simply add it to the XAML and hook the DrawFunction delegate. Add code to clear the backbuffer and run it. You should see the view drawn using whatever color you cleared to. You could also hook the DeviceCreated event of the GraphicsDeviceService (in your main window’s constructor) and create a SpriteBatch along with a Texture2D (using the new Texture2D.FromStream method) and draw that.

Having the rendering is one great step forward. However most people will want to load in some content to use. I started again with the WinForms Content Sample and started working on upgrading it to use the correct MSBuild APIs. A lot of what was valid for that sample is now considered deprecated so we need to update some of the code to get things working. The ErrorLogger and ServiceContainer are fine as they are, it’s the ContentBuilder we will draw our attention to.

First we have to update the xnaVersion constant to use the 4.0.0.0 version (same public key token) and I also added the rest of the pipeline assemblies:

 // What importers or processors should we load?
private const string xnaVersion = 
    ", Version=4.0.0.0, PublicKeyToken=6d5c3888ef60e27d";
private static readonly string[] pipelineAssemblies =
{
    "Microsoft.Xna.Framework.Content.Pipeline.AudioImporters" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.EffectImporter" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.FBXImporter" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.TextureImporter" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.VideoImporters" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.XImporter" + xnaVersion,
};

Next we want to remove all using statements for Microsoft.Build.* because the ones already there aren’t the ones we want to use. Instead, replace them with

 using Microsoft.Build.Evaluation;

Now head down into the fields and find the BuildEngine which we want to replace with a ProjectCollection. Project is still valid but it is now a Project type from a different namespace.

 // MSBuild objects used to dynamically build content.
private ProjectCollection projectCollection;
private Project msBuildProject;

With those changes we now have to fix up our methods one by one to get things building. First the CreateBuildProject method which creates the MSBuild project and fills in the basic information. If you compare the WinForms sample with this code, you’ll see it’s quite similar; there are just a few APIs that needed adjustment to fit everything in.

 /// <summary>
/// Creates a temporary MSBuild content project in memory.
/// </summary>
void CreateBuildProject()
{
    string projectPath = 
        Path.Combine(buildDirectory, "content.contentproj");
    string outputPath = Path.Combine(buildDirectory, "bin");

    // Create the project collection
    projectCollection = new ProjectCollection();

    // Hook up our custom error logger.
    errorLogger = new ErrorLogger();
    projectCollection.RegisterLogger(errorLogger);

    // Create the build project.
    msBuildProject = new Project(projectCollection);
    msBuildProject.FullPath = projectPath;

    // set up the properties we care about
    msBuildProject.SetProperty("XnaPlatform", "Windows");
    msBuildProject.SetProperty("XnaFrameworkVersion", "v4.0");
    msBuildProject.SetProperty("XnaProfile", "HiDef");
    msBuildProject.SetProperty("Configuration", "Release");
    msBuildProject.SetProperty("OutputPath", outputPath);

    // Register any custom importers or processors.
    foreach (string pipelineAssembly in pipelineAssemblies)
    {
        msBuildProject.AddItem("Reference", pipelineAssembly);
    }

    // Include the standard targets file that defines
    // how to build XNA Framework content.
    msBuildProject.Xml.AddImport(
        "$(MSBuildExtensionsPath)\\Microsoft\\XNA Game Studio\\v4.0\\" +
        "Microsoft.Xna.GameStudio.ContentPipeline.targets");
}

Next up is the Add method which adds a piece of content to our project. Again, nothing has really changed here; we’ve just restructured it around the new APIs for the Microsoft.Build.Evaluation.Project type.

 /// <summary>
/// Adds a new content file to the MSBuild project. The importer and
/// processor are optional: if you leave the importer null, it will
/// be autodetected based on the file extension, and if you leave the
/// processor null, data will be passed through without any processing.
/// </summary>
public void Add(
    string filename, string name, 
    string importer, string processor)
{
    // set up the metadata for this item
    var metadata = new SortedList<string,string>();
    metadata.Add("Link", Path.GetFileName(filename));
    metadata.Add("Name", name);

    if (!string.IsNullOrEmpty(importer))
        metadata.Add("Importer", importer);

    if (!string.IsNullOrEmpty(processor))
        metadata.Add("Processor", processor);

    // add the item
    msBuildProject.AddItem("Compile", filename, metadata);
}

Lastly we have Clear. Clear used to be a one liner, but the method it used doesn’t exist anymore. Instead we use LINQ to find all of the items and then remove them using a new RemoveItems method.

 /// <summary>
/// Removes all content files from the MSBuild project.
/// </summary>
public void Clear()
{
    // select all compiled objects in the project and remove them
    var compileObjects = from i in project.Items 
                         where i.ItemType == "Compile" 
                         select i;
    project.RemoveItems(compileObjects);
}

Now we’ve updated ContentBuilder to use .NET 4.0 and are ready to  go. From here on it’s exactly like the WinForms Content Sample in how you create the ContentBuilder, ContentManager, and so on, so I’ll just let you look it up there instead of pasting it all here.

And there we go. XNA Framework 4.0 rendering inside of a WPF application. Hopefully this helps you get up and running with your own apps leveraging these great technologies.