Blending Two Images in Real-Time

Article
11/17/2011

Recently, I was asked to whip up a WPF code sample that mimicked how Expression Design uses Alpha Blending to merge two images together (specifically, the Darken and Lighten blend modes). Instead of pre-processing the image in Expression Design, the developers wanted a programmable, real-time way to blend images in their application.

Since the application was using WPF 4.0, a pixel shader was a natural fit for this problem. There have a been a number of great blog posts about pixel shaders, HLSL, and WPF, so I won’t go into detail about the inner workings. The posts below are the main ones I used to get an understanding how to solve this problem.

While the above articles showed everything needed to get a working sample up and running, there was no overarching sample that brought everything together. Building a working WPF application that shows an image with a pixel shader effect tied to it is relatively straightforward. Adding a second image parameter to the pixel shader and an opacity factor is also very simple given the level of pixel shader support in WPF. One thing that you need to keep in mind is the fact that WPF uses pre-multiplied alpha values for optimization purposes. That affects the math that you need to do in the pixel shader.

My initial attempt worked great when the images were the same size, however when the images were different sizes, the second image was resized to the size of the base image. This again is an optimization done by WPF, but definitely doesn’t give the results we would want. Greg Schechter's last blog post in his series on GPU-based effects in WPF explains this in detail. To make the ViewBox technique work with absolute X, Y positioning coordinates, there is some sizing math that needs to be done. Since I try to keep as little code behind as possible in my views, I wrote the converter below to do all of this work for me.

     public class ViewboxRectConverter : IMultiValueConverter
    {
        public object Convert(object[] values, Type targetType, object parameter, CultureInfo culture)
        {
            if ((values == null) || (values.Length < 4))
            {
                throw new InvalidOperationException(
                    string.Format(
                    CultureInfo.InvariantCulture,
                    "Invalid number of parameters.  Found: {0}  Minimum Required: 4",
                    values.Length));
            }

            double baseImageWidth = values[0] == null ? 0.0d : (double)values[0];
            double baseImageHeight = values[1] == null ? 0.0d : (double)values[1];
            int blendImageWidth = (values[2] == null) ? 1 : (int)values[2];
            int blendImageHeight = (values[3] == null) ? 1 : (int)values[3];

            // Just a shortcut to save some if statements.
            double x = (values.Length > 4) ? values[4] as double? ?? 0.0d : 0.0d;
            double y = (values.Length > 5) ? values[5] as double? ?? 0.0d : 0.0d;

            double scaleX = baseImageWidth / blendImageWidth;
            double scaleY = baseImageHeight / blendImageHeight;

            return new Rect((x / blendImageWidth) * -1, (y / blendImageHeight) * -1, scaleX, scaleY);
        }

        public object[] ConvertBack(
            object value,
            Type[] targetTypes,
            object parameter,
            CultureInfo culture)
        {
            throw new NotImplementedException();
        }

The XAML usage is as follows.

 <ImageBrush x:Name="imageBrush" ImageSource="{Binding ElementName=blendImageName, Path=SelectedValue}">
    <ImageBrush.Viewbox>
        <MultiBinding Converter="{StaticResource rectConverter}">
            <Binding RelativeSource="{RelativeSource Mode=FindAncestor, AncestorType=Image}" Path="ActualWidth" />
            <Binding RelativeSource="{RelativeSource Mode=FindAncestor, AncestorType=Image}" Path="ActualHeight" />
            <Binding RelativeSource="{RelativeSource Mode=Self}" Path="(ImageBrush.ImageSource).(BitmapSource.PixelWidth)" />
            <Binding RelativeSource="{RelativeSource Mode=Self}" Path="(ImageBrush.ImageSource).(BitmapSource.PixelHeight)" />
            <Binding ElementName="xSlider" Path="Value" />
            <Binding ElementName="ySlider" Path="Value" />
        </MultiBinding>
    </ImageBrush.Viewbox>
</ImageBrush>

You might notice that I am binding to the actual width and height of the Image and ImageBrush respectively. This is to allow our converter to update appropriately when either image changes in our view. Binding to the ImageSource property doesn’t quite work the same as other bindings, so I took the path of least resistance.

You can download the sample application here. It contains free sample images from various sources on the web. I have resized all of the base images to 1024 x 768 and 96 ppi. If you want to add your own, just make sure that it is 96 ppi, or WPF will resize it automatically.

To make this sample application build, you will need to install the Microsoft DirectX SDK. I used the June 2010 DirectX SDK which is available here, but as long as you have a version that can compile Pixel Shader 2.0 FX files, it should be fine. Also, to make my sample easier for me to test, there are two pre-build events that compile the pixel shaders every time you build. My install path may be different than yours, so if you get errors during a build, check the path to the fxc.exe and make sure it is pointing to your install of the DirectX SDK.

There are many cool things that can be done with pixel shaders and this basic setup should allow you to add your own to try out. For a great set of shaders, check out the WPF Pixel Shader Effects Library on CodePlex.

Blending Two Images in Real-Time

Additional resources