# How Shawn learned to stop worrying and love premultiplied alpha

I like to think I know what I'm doing with computer graphics, so I find it embarrassing that it took us four versions to get something as fundamental as alpha blending right.

To cut a long story short, I didn't properly understand alpha blending until way too recently ðŸ™‚

(but in my defense, it seems like many other people don't understand it either)

To leave the long story long:

Graphics programming, especially for games, is an interesting mix of science, art, and Rube Goldberg contraption. Some things can be mathematically proven to have a single correct solution, so you'd be crazy to do them any other way. Other times we can achieve great results from ad-hoc hacks that only make sense for a specific game or piece of hardware.

When I started making games, the focus was very much on the hack side of things. Most projects began with being given a piece of hardware, and told to go off and make it do something cool. Few people had formal backgrounds in math or computer graphics.

Over time, the focus shifted toward properly understanding the theory behind what we were doing. Many projects began with being given a stack of SIGGRAPH papers, and told to go off and implement the techniques described therein. This often left those of us who grew up figuring things out by trial and error scrambling to relearn old tricks.

So how did I learn about alpha blending?

The first time I encountered the concept of blending colors was in college, where I wrote a game programming library called Allegro. Working with 8 bit palettized colors, this included a draw_trans_sprite() function which used a 64k lookup table to precompute every possible combination of two color values. This could achieve many varied effects just by changing what went into the table, but with only 256 colors to choose from, the quality of the results was, well, kinda dubious ðŸ™‚

The first time I experienced proper hardware alpha blending was on the Nintendo 64, where we used a mixture of additive and interpolative blending, mostly for special effects. Textures were extremely small, usually just 4 bpp, and drawn by hand with Deluxe Paint. If we saw filtering or blending artifacts, we just asked the artist to change the offending texture.

The first time I ran into alpha blending problems was on Playstation II, where the artists used scanned photos as a source for creating 4 and 8 bpp palettized textures. Thanks to the unholy combination of filtering and interpolative alpha blending, we got ugly fringes around the edges of our tree billboards. I fixed this by using our equivalent of a custom content processor to automatically adjust the RGB of transparent palette entries, changing them to match the average color of the texture as a whole. Problem solved (or so I thought...)

The second time I ran into alpha blending problems was when we moved from Playstation II to the original Xbox, and switched from palettized textures to the more efficient DXT compression. DXT1 only supports transparent pixels where the RGB is zero, so my automatic fixup was no longer possible and the black fringes returned. I 'fixed' this by switching to DXT5 format wherever possible, plus using alpha test to skip drawing the problematic borders of textures for which DXT5 would use too much memory. This was a delicate balancing act. If the alpha test threshold was too high, the trees appeared jagged and hard edged, but it was too low, they had black borders. Somewhere in between, both problems still existed, but neither was too obvious. Problem solved? (kinda, but it made me feel dirty...)

The third time I ran into alpha blending problems was when I optimized particle rendering by drawing into a lower resolution rendertarget. Rendertarget composition is fundamentally impossible when using interpolative alpha, but I didn't know this at the time. So I sat down with pen and paper, did some algebra, and figured out the blend math necessary to get the result I wanted. It turns out that I independently reinvented premultiplied blending, but I failed to generalize this or realize how useful it could be for other things!

A few years later, I read Tom Forsyth's excellent article about premultiplied blending. "Aha!" quoth I. "So that's what that crazy thing I used for the particles is called. And wow, it seems pretty handy for other stuff too." Then I promptly forgot all about it.

A few more years later, I left Climax, left England, moved to Seattle, joined Microsoft, and started working on XNA. Having forgotten Tom's article, it never occurred to me (or I guess to anyone else here) that this might be something worth including in our API.

Shortly after we shipped XNA v1, I saw the first "wtf is up with these crazy borders around my sprite?" question on our forums. In the middle of typing up a lengthy "here's the laundry list of hacks I used to work around this problem in MotoGP" reply, I remembered, "wait, didn't Tom know a better way to fix this?"

After several years of getting this same question, pointing people toward Tom's article, and watching the ensuing confusion because it was too hard to do this with our API, we decided it was time to fix this mistake once and for all.

Enter Game Studio 4.0.

And they all lived happily ever after.

(touch wood)

Makes me wonder what else is out there that I don't know but should?

1. Cool post ðŸ™‚

I never knew that you made allegro… how long did you work on that for?

2. Matt says:

3. I love these small story like parts in between the tech articles. I so recognize myself in there sometimes (most techies will).

I have an equal story, but about developing database driven applications. (I was once asked to make one when I was 17, and all I had done was toying with VB6, after that one when I was just studying CS for 6 months, and one after a few years of study, can you gues the quality? ðŸ™‚ ).

4. PolyVector says:

With all this talk about premultiplied alpha lately, I can’t help but "worry". ðŸ™‚

Isn’t PMA only practical for a limited range of (lightly colored / additively blended) things like clouds or fire?  I mean, black can’t be drawn at all, colors wash out and turn transparent, etc…

I think it’s a great idea making PMA easier to use, but making it default seems so strange to me.  I hate repetitive/rude support questions, which is why I selfishly stopped supporting RocketDock, so I do sympathize… I just think PMA makes more sense as an option, otherwise your questions will just turn into "Why is my frickin’ tree transparent?!? The alpha is 255 in Photoshop!"  And honestly, that’s a valid complaint compared to inherent alpha/edge issues that are beyond the scope of what XNA can/should handle.

Just my silly 2 cents… You did write Allegro, so you should probably just ignore me. ðŸ˜‰

5. PolyVector says:

For some reason I assumed the alpha channel was dropped so only additive blending was possible.  At least I was right about one thing: I should be ignored. *hits head* ðŸ™‚

6. rotoglup says:

Just out of curiosity (I’m not an XNA user), is there any gotcha in using premultiplied alpha and sRGB colors ? Does XNA content processor handle this case ?

7. David Black says:

Pre-multiplied alpha is cool, but I wouldnt go as far as shawn and give the impression it is the solution to all alpha blending problems:-)

There are plenty of other alpha tricks, eg fading alpha based on depth, fading based on depth to object in depth buffer(ie soft particles), sorting meshes into a set of fixed directions(grass), sorting into shells(eg hair layers) etc etc

It is a shame that standard graphics texts do not cover this type of thing very well(I cant even remember any references from Foley and Van Dam, or the more recent real time rendering)

I too learned about pre multiplied alpha(and lots of other neat tricks) from posts by TomF etc…

8. ShawnHargreaves says:

> I never knew that you made allegro… how long did you work on that for?

Pretty much the whole time I was in college, when I should have been studying for my degree, I was writing Allegro code instead ðŸ™‚

But hey, that’s how I learned to program, so I figure it paid off in the end!

After I graduated and got a job, I maintained it for a couple more years, mostly just integrating changes from other people rather than writing huge new things myself. Then I gradually phased out my involvement and handed the project over to other maintainers (who have been doing a great job keeping it going ever since).

9. ShawnHargreaves says:

> Isn’t PMA only practical for a limited range of (lightly colored / additively blended) things like clouds or fire?  I mean, black can’t be drawn at all, colors wash out and turn transparent, etc…

I think you are confusing premultiplied with additive blending. Additive has all the characteristics you describe here, which makes it great for specific effects (fire, magic, glows) but useless for anything elsee.

Premultiplied blending is a direct algebraic rearrangement of interpolative blending, so anything you can do with interpolative colors, you can do with premultiplied as well, just with less artifacts.

10. ShawnHargreaves says:

> is there any gotcha in using premultiplied alpha and sRGB colors ?

Premultiplied and sRGB can work together, but (like pretty much anything involving sRGB formats) it does require some care to get the right results.

However, XNA doesn’t currently support sRGB at all.

11. ShawnHargreaves says:

> Pre-multiplied alpha is cool, but I wouldnt go as far as shawn and give the impression it is the solution to all alpha blending problems:-)

I wouldn’t go that far either! I just think premultiplied is a good default place to start (certainly much better that interpolative alpha).

> There are plenty of other alpha tricks, eg fading alpha based on depth, fading based on depth to object in depth buffer(ie soft particles), sorting meshes into a set of fixed directions(grass), sorting into shells(eg hair layers) etc etc

Oooh oooh – methinks you need to write articles about these techniques! That’s an awesome list of blog content right there ðŸ™‚

12. David Black says:

>>> Pre-multiplied alpha is cool, but I wouldnt go as far as shawn and give the impression it is the solution to all alpha blending problems:-)

I wouldn’t go that far either! I just think premultiplied is a good default place to start (certainly much better that interpolative alpha).

<<<

I guess often pre-multiplied alpha is touted as a way to avoid sorting etc. Which it isnt really, although often the results can be OK for some things.

>>> There are plenty of other alpha tricks, eg fading alpha based on depth, fading based on depth to object in depth buffer(ie soft particles), sorting meshes into a set of fixed directions(grass), sorting into shells(eg hair layers) etc etc

Oooh oooh – methinks you need to write articles about these techniques! That’s an awesome list of blog content right there ðŸ™‚

<<<

Well I tend to prefer writing code to blogging non stop:-P

Anyhow I just wrote an entry on why the standard light attenuation function(D3D/OGL fixed function) sucks….

Which is a good example of why even having only simple programmable shaders is a good thing.

13. i said it before and i’ll say it again, investigate QFloats for premultiplied alpha. brings the premultiplied component down to 1 byte (q = 7) – as well as being faster during composition.  it does have a (impercievable) decrease in quality.

real trick is to get the gpu to accept it.

14. David Black says:

So whats a QFloat? The alpha is normally 8bits anyway, so what do they gain? Is it some kind of compression for HDR data perhaps?

Perhaps you could also write a blog entry about QFloats…? ðŸ™‚

15. jwatte says:

Some more while I’m at it:

Texture processor:

– Why is color key on the default?

– Why is mip maps off the default?

16. jwatte says:

It’s great to finally see you join the club! Thank you!

Some more things:

– matrices for animation keyframes? Volume collapse!

– matrices multiply from the left, quaternions from the right? Wtf?

– no procedural audio? (Wait, you’re fixing this, too in 4! Yay!)

– left-handed triangle culling, right-handed coordinate system? (I mean, really?)

Still time for 4.0 release ðŸ™‚

17. ShawnHargreaves says:

> Texture processor:

> – Why is color key on the default?

> – Why is mip maps off the default?

Because these are the best defaults for the largest number of developers (and especially for beginner developers who are less likely to understand these options or know how to change them).

When we look at what people use textures for, significantly more than half are used as sprites in a 2D scene, often with non-pow-2 textures, where mipmaps are both unnecessary and would not even work on much hardware. For these people, no mips is a better default.

Sure, people making 3D games will want mips, but a) this is less common, b) 3D developers are typically more experienced than those working in 2D, so are more able to change away from the default, and c) many 3D developers build their textures indirectly via ModelProcessor, rather than adding them directly to the content project, and in that case mipmaps on is indeed the default.

Likewise, color key on is a good default for the many beginner developers who do not understand alpha channels and do not have a paint program that knows how to create them. The more experienced folks who have Photoshop, know how to use it, and want to use magenta in their textures, are more able to change this default than the beginner who is using Microsoft Paint and does not yet know what an alpha channel even is ðŸ™‚