Need Help with my Chroma Key Pixel Shader Algorithm

I’ve just uploaded a simple demo application that shows the Silverlight 3 Chroma Key Effect in action.  If you don’t have the Silverlight 3 beta installed you will need to install it from here (the install link in the application will not work). 


If you watch the demo, you’ll notice ghosting along the edges of the dancers – obviously I need help with my pixel shader algorithm:

float4 PS( VS_OUTPUT input ) : SV_Target
     float4 color = tex2D( ImageSampler, input.UV );
     if (abs(InputColor.r - color.r) <= Tolerance && 
         abs(InputColor.g - color.g) <= Tolerance &&
         abs(InputColor.b - color.b) <= Tolerance)
      color.rgba = 0;
   return color;

If anyone has suggestions on how to make this algorithm better – or has a better algorithm, please share it with me.

Thanks in advance.

Comments (8)

  1. Ted Howard says:

    I would try converting to HSV color space in the pixel shader and having a separate tolerance for each component (use a float3 constant and there’s no perf diff). You can either calculate it or use a texture as a lookup table (not sure about Silverlight 3’s texture/shader compat). HSV is more intuitive and should give a better distance function. There are other color spaces you could try. YUV might work well but it’s designed to deemphasize chrominance.

    You should probably use a continuous function for alpha. Anything but a step should give a nice improvement.

  2. In this Issue: Erik Mork , Mark Monster , Steve Strong , Rishi , Kirupa Chinnathambi ( 2 ), Timmy Kokke

  3. Very very nice! 🙂  I attempted a WPF version a while ago, but HLSL / shader model 2.0 didn’t allow enough statements to accomplish RGB -> HSV -> RGB conversion.  Wonder if it could be done with multiple shaders + inputs somehow? (or maybe some close approximation of HSV?).  I’ll be watching this with great interest 🙂

  4. Ted Howard says:

    Relative luminance ( is a linear conversion from RGB, but it just makes the threshold more intuitive.

    Preprocessing is the way to go ideally, but I’m sure that’s not an option. In general, for big calculations in shaders, you try using a texture as a look-up-table for parts of the calculations that won’t fit in shader limits, but I don’t think that helps here.

  5. Forrestshi says:

    I have create one very complicated algorithm, and here are some effects

    I am not familiar with sliver** tech, not sure if can port to sliver.

    One point is that alpha channel should have more continue value from 0 to 255 , not only two sort of values: 0 , 255.

    Welcome for more discussions with email :