Pixel Formats (Part 1: Unsigned Integers)

This is the first installment of what I plan to be an ongoing series of personal technical notes about all things related to Windows Media Photo. We’re starting with the very basics – pixel formats.

Windows Media Photo supports far more pixel formats than any other image file format. In fact, several of the supported pixel formats are brand new, being introduced for the first time with Windows Vista (and WIC or .NET Frameworks 3.0 for down-level support.) Let’s take a closer look at pixel formats and the new capabilities supported by Windows Media Photo.

A Little Background

Historically, color images have been encoded using three unsigned integer values for red, green and blue. Monochrome (or gray scale) images use a single unsigned integer per pixel. Virtually all consumer photography uses 8-bit unsigned integer values, providing 256 unique values for each of the three channels. For more demanding requirements, many applications also support 16-bit unsigned integers, offering 65536 unique values per channel.

256 steps from the darkest value to the lightest value present some significant limitations. In actuality, 256 steps per channel are all that are required to display an image on a monitor. But that assumes that the values correspond to the monitor’s interpretation of color. Since that is rarely the case, transformations between different color spaces are required, losing a certain amount of the already limited precision. The artifacts that come from this loss of precision often shows up as visible bands in smooth, continuous color areas, where there are no longer enough unique values to represent the color range without seeing visible steps between adjacent values. Using 16 bits per channel provides more than sufficient resolution to insure more than 8 bits per channel are always retained through any color space conversion. Of course, twice as much data is required to guarantee this quality.

Using unsigned integers is currently the most common way to encode color information, and Windows Media Photo has rich support for unsigned integer pixel formats in multiple color formats and bit depths.

The Colorless World of Gray Scale

Let’s start with the Gray unsigned integer pixel formats:

PixelFormat Name

GUID

Ch

BPC

BPP

Num

Color

A

B

WICPixelFormat8bppGray

6fddc324-4e03-4bfe-b1853d77768dc908

1

8

8

UINT

Gray

ü

WICPixelFormat16bppGray

6fddc324-4e03-4bfe-b1853d77768dc90b

1

16

16

UINT

Gray

ü

WICPixelFormatBlackWhite

6fddc324-4e03-4bfe-b1853d77768dc905

1

1

1

UINT

Gray

If you’ve already taken a look at the Windows Media Photo Feature Specification, then you’re familiar with these pixel format tables. But, let’s recap the information summarized in the table.

Name This is the descriptive name that we use to refer to the pixel format. This is frequently abbreviated by dropping the WICPixelFormat prefix.

GUID The 128-bit Globally Unique Identifier for the pixel format.

Ch The number of channels.

BPC Bits per channel.

BPP Bits per pixel. This is typically the number of channels multiplied by the number of bits per channel. However, some pixel formats contain unused information in each pixel value to align pixel values to a specific boundary.

Num The numerical format.

Color The color format.

A Indicates that the pixel format contains an alpha channel. This is included in the channel count.

B Indicates that this pixel format is one of the BASIC pixel formats that is required to be supported by all Windows Media Photo decoders.

The unsigned integer Gray pixel formats are pretty straightforward. The two Basic formats are either 8bpc or 16bpc. These correspond to the most commonly used monochrome pixel formats in use today. In addition, the third pixel format uses a single bit per pixel for black/white images.

The Most Common Color Format - RGB

Windows Media Photo supports an extensive set of unsigned integer RGB pixel formats:

The Basic formats include RGB in both 8bpc and 16bpc, plus 8bpc in BGR channel order.

RGB Channel Ordering

Let’s take a little detour to discuss channel ordering. The RGB color format is the only one where we have to deal with different channel ordering. Windows Media Photo’s support for different channel orders is largely for legacy reasons, providing support for the most commonly used ways of representing color. Ideally, channels would always be organized in the same order, simplifying the pixel format options. However, the existence of big-endian and little-endian processor architectures has led to different approaches for channel order of RGB data, requiring ongoing support for backward compatibility.

Historically, the industry has been inconsistent in the names for pixel formats compared to the specific ordering of channel values in an uncompressed image bit stream. We’ve made every effort to be consistent with Windows Media Photo. The pixel format name specifies the channel ordering as they appear in the sequential stream of uncompressed image data. Because of differences with big endian vs. little endian processor architectures, this becomes a little more confusing with channel values represented by 8bpc or less.

For a bitmap in 8bpc RGB pixel format (WICPixelFormat24bppRGB), the first byte in the uncompressed data stream is the red channel for the first pixel, followed by the green channel and then the blue channel. This is the typical way that 8bpc RGB data is stored in an uncompressed TIFF file. WICPixelFormat24bppBGR stores the channels in the reverse order, starting with the blue channel. This is the channel order used in 8bpc BMP files.

When a fourth channel is added to 8bpc pixel formats (either an alpha channel or an unused padding byte to align pixels on 32-bit word boundaries), the channel order of the bytes within a 32-bit word on a little-endian system are typically interpreted (and named) in the opposite order as they appear in the uncompressed data stream. However, with WIC pixel formats, the naming always references the channel order in the data stream, not how they might be reorganized within a larger pixel word on a little-endian system.

In addition to the Basic pixel formats, the set of unsigned integer RGB pixel formats include versions with normal and pre-multiplied alpha channels, an 8bpc version with an unused padding byte, and three different packed bit formats. We’ll save the discussion of alpha channels for another day.

WICPixelFormat32bppBGR is very similar to WICPixelFormat32bppBGRA; the only difference is that the latter uses the value in the fourth byte as an alpha channel while the former ignores it. These two pixel formats are very useful for storing an uncompressed pixel in a 32-bit word. The low order byte of the word is the blue channel, the next higher byte is green, then red, and the high order byte is either the alpha channel or unused, depending on the pixel format.

Packed Bit RGB Pixel Formats

Windows Media Photo supports three unsigned integer RGB packed-bit formats. These are formats where the bits per channel are not a multiple of 8-bit bytes. Because the channel values aren’t byte-aligned, the uncompressed data stream must be viewed as a sequence of pixel values rather than a sequence of channel values. The order of the channels within a pixel are defined within a word, and therefore, are dependent on whether the word is based on a big endian or little endian architecture.

Windows Media Photo always assumes a little endian (Intel/AMD) architecture, and the packed bit channel values are always organized accordingly. Therefore, a big-endian system must reorder the bytes within a word before interpreting the channel values.

WICPixelFormatBGR555 stores three 5-bit channel values in a single 16-bit word, with an unused bit. The least significant 5 bits are the blue value; the next 5 bits are the green value, followed by the red value. The most significant bit is unused. WICPixelFormatBGR565 increases the green channel to six bits, so all bits in a 16-bit word are used. These pixel formats are commonly used in hardware implementations, including displays in mobile devices. Windows Media Photo allows direct encoding in the device’s native pixel format.

WICPixelFormatBGR101010 stores three 10-bit channel values in a 32-bit word, with two unused bits. The least significant 10 bits are the blue value; the next 10 bits are the green value, followed by the red value. The most significant two bits are unused. This is a very useful format for increasing the precision of unsigned integer image data without a significant increase in uncompressed bitmap size. However, it can often be cumbersome to deal with 10-bit values, and the additional processing to extract the values outweighs advantages of saving storage space.

Support for Photo Printing

Windows Media Photo supports multiple pixel formats specifically designed for the needs of image printing. This includes numerous pixel formats in CMYK and n-Channel color formats.

Here are the CMYK pixel formats supported by Windows Media Photo:

CMYK has historically been used for encoding information for printing rather than display on a monitor. However, virtually all PC photo printers are actually RGB, not CMYK devices. These printers use multiple inks that don’t directly map to CMYK channels. A printer expects to receive image content in RGB color format, and translates directly to the color space defined by the printer’s specific ink configuration. Only conventional offset printing uses CMYK channels, and the CMYK color space is supported for these printing scenarios. Windows Media Photo supports CMYK image data in 8bpc and 16bpc bit depths, with and without an alpha channel. I’ll save a more detailed look at the use of CMYK pixel formats for another time.

Windows Media Photo has been designed to support a number of printing scenarios, including native support for printer color spaces. Support is provided for up to eight channels of arbitrary image data, making it possible to encode images in the color space defined by the printer’s ink configuration. These n-channel formats (listed below) are provided in either 8bpc or 16bpc bit depths, either with or without an alpha channel. We’ll discuss n-channel pixel formats in much more detail in a future installment.

That’s a summary of the unsigned integer pixel formats. Along the way we had a chance to talk about bit depth and channel ordering. We skipped over alpha channels (and pre-multiplied alpha channels) and we’ve only scratched the surface on CMYK and n-channel pixel formats. We also haven’t talked at all about color space and color profiles, which are very important for unsigned integer pixel formats. We’ll get to all these topics in much more detail in future installments.

Coming Up Next

While unsigned integer with RGB color formats is the legacy encoding for images, it has numerous shortcomings. When encoding an image using unsigned integers, the smallest value (0) represents black, and the largest value represents maximum saturation for that channel. When all three channels are set to the maximum value, the resulting color is white.

However, these minimum or maximum values are based on the specific color context being used. When converting between color spaces, any color values that exceed the limits of either color space are lost because we can only encode values that reside inside the representative bounds of each color space. The fact that we must always clamp data to fit within the current color space is one of the principal ways that useful image content is destroyed during image processing workflows. This has led to complicated (and usually proprietary) workflows to edit in source color spaces to minimize this data loss.

In our next installment we’ll investigate how we can use fixed point or floating point numerical encoding to retain information that would otherwise be destroyed by clipping to the current color space. This provides a much more practical way to preserve all the image information with much less complex workflows.

Send your Feedback

This is the first of what I plan to be an ongoing series of technical notes on all aspects of Windows Media Photo and related topics on image processing and workflows. I’d really appreciate your feedback. I’m using my personal blog to share this information as directly and timely as possible. Eventually, we’ll organize all this information into a comprehensive technical document. In the mean time, this blog allows me to share it now, but it also means that it hasn’t gone through the same level of editing and review that we use for our more official publications. My blog writing style will be a little more freeform, and I’m sure the minimal proofreading and reviews will allow a number of errors and typos to sneak through.

So send me your feedback to help me make this blog as helpful as possible. Are these technical notes useful? Is this level of presentation too simplistic or does it include too much technical details that aren’t adequately explained? What topics are you most interested in? What bugs and errors do you see? You can reply using the comments, below, or send me email with the link on the left.

I look forward to hearing your feedback!