The evolution of the ICO file format, part 1: Monochrome beginnings


This week is devoted to the evolution of the ICO file format. Note that the icon resource format is different from the ICO file format; I'll save that topic for another day.

The ICO file begins with a fixed header:

typedef struct ICONDIR {
    WORD          idReserved;
    WORD          idType;
    WORD          idCount;
    ICONDIRENTRY  idEntries[];
} ICONHEADER;

idReserved must be zero, and idType must be 1. The idCount describes how many images are included in this ICO file. An ICO file is really a collection of images; the theory is that each image is an alternate representation of the same underlying concept, but at different sizes and color depths. There is nothing to prevent you, in principle, from creating an ICO file where the 16×16 image looks nothing like the 32×32 image, but your users will probably be confused.

After the idCount is an array of ICONDIRECTORY entries whose length is given by idCount.

struct IconDirectoryEntry {
    BYTE  bWidth;
    BYTE  bHeight;
    BYTE  bColorCount;
    BYTE  bReserved;
    WORD  wPlanes;
    WORD  wBitCount;
    DWORD dwBytesInRes;
    DWORD dwImageOffset;
};

The bWidth and bHeight are the dimensions of the image. Originally, the supported range was 1 through 255, but starting in Windows 95 (and Windows NT 4), the value 0 is accepted as representing a width or height of 256.

The wBitCount and wPlanes describe the color depth of the image; for monochrome icons, these value are both 1. The bReserved must be zero. The dwImageOffset and dwBytesInRes describe the location (relative to the start of the ICO file) and size in bytes of the actual image data.

And then there's bColorCount. Poor bColorCount. It's supposed to be equal to the number of colors in the image; in other words,

bColorCount = 1 << (wBitCount * wPlanes)

If wBitCount * wPlanes is greater than or equal to 8, then bColorCount is zero.

In practice, a lot of people get lazy about filling in the bColorCount and set it to zero, even for 4-color or 16-color icons. Starting in Windows XP, Windows autodetects this common error, but its autocorrection is slightly buggy in the case of planar bitmaps. Fortunately, almost nobody uses planar bitmaps any more, but still, it would be in your best interest not to rely on the autocorrection performed by Windows and just set your bColorCount correctly in the first place. An incorrect bColorCount means that when Windows tries to find the best image for your icon, it may choose a suboptimal one because it based its decision on incorrect color depth information.

Although it probably isn't true, I will pretend that monochrome icons existed before color icons, because it makes the storytelling easier.

A monochome icon is described by two bitmaps, called AND (or mask) and XOR (or image, or when we get to color icons, color). Drawing an icon takes place in two steps: First, the mask is ANDed with the screen, then the image is XORed. In other words,

pixel = (screen AND mask) XOR image

By choosing appropriate values for mask and image, you can cover all the possible monochrome BLT operations.

mask image result operation
0 0 (screen AND 0) XOR 0 = 0 blackness
0 1 (screen AND 0) XOR 1 = 1 whiteness
1 0 (screen AND 1) XOR 0 = screen nop
1 1 (screen AND 1) XOR 1 = NOT screen invert

Conceptually, the mask specifies which pixels from the image should be copied to the destination: A black pixel in the mask means that the corresponding pixel in the image is copied.

The mask and image bitmaps are physically stored as one single double-height DIB. The image bitmap comes first, followed by the mask. (But since DIBs are stored bottom-up, if you actually look at the bitmap, the mask is in the top half of the bitmap and the image is in the bottom half).

In terms of file format, each icon image is stored in the form of a BITMAPINFO (which itself takes the form of a BITMAPINFOHEADER followed by a color table), followed by the image pixels, followed by the mask pixels. The biCompression must be BI_RGB. Since this is a double-height bitmap, the biWidth is the width of the image, but the biHeight is double the image height. For example, a 16×16 icon would specify a width of 16 but a height of 16 × 2 = 32.

That's pretty much it for classic monochrome icons. Next time we'll look at color icons.

Still, given what you know now, the following story will make sense.

A customer contacted the shell team to report that despite all their best efforts, they could not get Windows to use the image they wanted from their .ICO file. Windows for some reason always chose a low-color icon instead of using the high-color icon. For example, even though the .ICO file had a 32bpp image available, Windows always chose to use the 16-color (4bpp) image, even when running on a 32bpp display.

A closer inspection of the offending .ICO file revealed that the bColorCount in the IconDirectoryEntry for all the images was set to 1, regardless of the actual color depth of the image. The table of contents for the .ICO file said "Yeah, all I've got are monochrome images. I've got three 48×48 monochrome images, three 32×32 monochrome images, and three 16×16 monochrome images." Given this information, Windows figured, "Well, given those choices, I guess that means I'll use the monochrome one." It chose one of images (at pseudo-random), and went to the bitmap data and found, "Oh, hey, how about that, it's actually a 16-color image. Okay, well, I guess I can load that."

In summary, the .ICO file was improperly authored. Patching each IconDirectoryEntry in a hex editor made the icon work as intended. The customer thanked us for our investigation and said that they would take the issue up with their graphic design team.

Comments (36)
  1. Sven says:

    I wonder, what is the historical reason for including bColorCount if its value can be calculated from the wPlanes and wBitCount fields? I'm guessing that some older version of this field may not have had the wPlanes and wBitCount fields?

    [The reason will become evident when we look at the format of icon resources at some unspecified point in the future. -Raymond]
  2. Ben Hutchings says:

    How is bColorCount supposed to be set when the number of bits per pixel is >= 8 ?

  3. Karellen says:

    "The customer thanked us for our investigation and said that they would take the issue up with their graphic design team."

    Um, yeah, because of course graphic designers should know how to set the bColorCount in the IconDirectoryEntry of ICO files correctly!

    /sarcasm

    Did you point out they'd probably be better off instead talking to the people who wrote the tools their graphic designers are using? The designers should be hitting "save" and have it all Just Work(tm).

    [I trusted that they were using the appropriate shorthand. -Raymond]
  4. Gabe says:

    One also has to wonder by bColorCount isn't just ignored completely. Since the value only holds up to 255, anything parsing the structure has to look at wPlanes and wBitCount anyway, so why not just look there to begin with?

  5. configurator says:

    @Ben: Raymond clearly said it: If wBitCount * wPlanes is greater than or equal to 8, then bColorCount is zero.

  6. Adrian says:

    This should be a fun series.  I've rummaged around inside ICO files before, but the documentation is scattered and doesn't put the details in the context of the evolution.

    @sven:  I think the color count is in case you use fewer than the maximum number of colors that the format would allow.  For example, if you had 4bpp, you could have 16 colors, but maybe you use only 7 of them.  You'd set the color count to 7 and need to provide only 7 entries in the color table.

  7. Falcon says:

    @Adrian: I think you missed this part:

    "Oh, hey, how about that, it's actually a 16-color image. Okay, well, I guess I can load that."

  8. Troll says:

    Great now will the shell team please ship a hotfix for the 16-bit icons not showing in 32-bit Windows 7 problem or it may never get fixed if Windows 8 ships in 64-bit only version. Is it or is it not going to before NTVDM is killed completely by shipping Windows 8 in x64 only?

    [It was removed on purpose. -Raymond]
  9. Troll says:

    I have read that but I wish it was back and working. It bothers me extremely for hundreds of 16-bit games. Even the Windows 7 taskbar treats 16-bit apps inferiorly (separate icon when run). I really wish at least 32-bit Windows treated 16-bit apps properly. Too many things are being removed since Vista arbitrarily.

    [The removal was not arbitrary. The icon extraction code was a liability, as noted in the article. Not mentioned in the article was that reimplementing the NE file parser was also a security liability. (Not a purely theoretical exercise. Custom file parsers have been the source of security vulnerabilities in the past.) -Raymond]
  10. Crescens2k says:

    That is because 16 bit application support is being removed from windows entirely, 64 bit versions of windows can't directly run 16 bit applications, and if you use server as the way of the future then the next client version will be 64 bit only.

    If you hate how it is being treated though, why not just use XP mode anyway, that will have everything you want and it works on both 32 and 64 bit versions of 7.

  11. Troll says:

    Well just let me tell you it degrades the experience for 16-bit apps. Is there any way I can write my own parser for only New Executables and make Explorer show them automatically?

    @Crescens2k, you are talking about 64-bit Windows while Windows 7 and Vista are also available in a pure 32-bit version where 16-bit support is no different from XP except for the icon issue.

  12. Dave Bacher says:

    The idea behind the catalog is to speed the selection and loading.

    First — the index is corrupt with the wrong number of colors.  What makes anyone think that the bits-per-plane and number of planes are any more accurate?  If they're wrong, Windows doesn't care either.  Just saying — all Windows cares about from a "does something end up on the screen" is that the DIB is valid.

    Second — if I were implementing this, most likely I'd pack the first 4 bytes into a DWORD and then pack the second 4 bytes into another DWORD, and run a bunch of subtracts in a loop.  Closest to zero wins.  for monochrome, I could see short circuting that and only doing the first DWORD — which in turn would cause the exact issue described in Raymond's post if the file was tagged monochrome, but actually had color data.

    Also coincidentally, if there were a 16, 256 and 65536 color icon, it would consistently pick the 16 color icon (it's closest to 0).

    Not saying that's what Windows is actually doing — just saying it's an easy/fast implementation that requires very little code, that happens to coincide with observed behavior.

    Also coincidentally, it would favor monochrome icons if a 4bpp image had the number of colors set to 0, since the monochrome would be closer to the target value.

  13. Nick says:

    "Too many things are being removed since Vista arbitrarily."

    I agree completely!  I mean, look at Scraps for example.  They were just up and ripped out without even offering a replacement. Windows is unusable without scraps!

  14. J says:

    "Well just let me tell you it degrades the experience for 16-bit apps."

    Who uses 16-bit apps anymore? This is a serious question, I really can't think of any uses other than legacy commercial stuff. And any decently competent IT team could implement the "package the icons in a 32-bit resource instead" solution if it was that important.

  15. Cheong says:

    @J: Well, how about that "How come the icons on toolbar of my old 16-bit application all look the same?"

    Add more fun if there's neither small text nor tooltips on the toolbar. :P

    [The icons on the toolbar of your old 16-bit application looks fine because it's being displayed by a 16-bit application. What got removed was 32-bit applications loading icons from 16-bit modules. If you want your 32-bit application to load icons from 16-bit modules, you are welcome to write your own parser that reads icons from 16-bit modules. -Raymond]
  16. Cheong says:

    Oh, I thought the ability to read 16-bit icons were taken away because you mentioned about "security liability". :O

    If it's just about the shortcut/application icons, there's really no big deal then.

  17. Cheong says:

    s/16-bit icons/16-bit module icons/

  18. Troll says:

    Just because few people use 16-bit apps doesn't make it a good excuse to degrade the user experience. Now MS will make the same maintenance cost vs value vs time argument. This makes them look like DOS apps: 1.bp.blogspot.com/…/WEP7.png Please fix this before at least for the last shipping 32-bit release of Windows.

  19. Barbie says:

    @Troll: you do take your name seriously. It does make it an excuse if very few people use it. A good one at that. The cost of maintenance outweighs the benefit to those few users, compared to to other features. So go ahead and write a shell extension if you really want it. It is possible to change the icons explorer shows. Here's a pointer:

    msdn.microsoft.com/…/cc144110%28VS.85%29.aspx

  20. Medinoc says:

    Damn, my post about the difficulty in writing an IconHandler got eaten (probably because I submitted after something expired and closed the tab without checking). So I'll make it brief:

    It's probably possible, but difficult, since one has to implement two behaviors based on whether the .exe is NE or PE. Maybe it's possible to piggyback on the default implementation in the case of PE files, but I'm not sure.

  21. Antonio Rodríguez says:

    It would be easier to build a small utility that, when handled a shortcut, checks if it points to an NE executable, and then extracts its icon next to the executable (or to a subfolder in AppData if unelevated) and permanently modifies the shortcut to use the extracted icon. Since both the taskbar and the Start menu use shortcuts to represent programs, it would solve the problem simply and nicely.

    The next step would be an utility that, when run, scanned through all taskbar and Start menu items, replacing its icon if necessary. Bonus points if it runs in the background and does it automatically when the system notifies that the Start menu or Quick launch folders have been modified :-) .

  22. Antonio Rodríguez says:

    It would be easier to build a small utility that, when handled a shortcut, checks if it points to an NE executable, and then extracts its icon next to the executable (or to a subfolder in AppData if unelevated) and permanently modifies the shortcut to use the extracted icon. Since both the taskbar and the Start menu use shortcuts to represent programs, it would solve the problem simply and nicely.

    The next step would be an utility that, when run, scanned through all taskbar and Start menu items, replacing its icon if necessary. Bonus points if it runs in the background and does it automatically when the system notifies that the Start menu or Quick launch folders have been modified :-) .

  23. asf says:

    @Medinoc: SHCreateDefaultExtractIcon maybe

  24. Anon says:

    @DWalker59:

    Is anyone still making new 32-bit processors and chipsets?

    I believe all current ARM processors are still 32-bit. And ARM processors are very popular.

  25. Frode Aarebrot says:

    Anon: So what? Windows doesn't run on ARM so your point is moot.

  26. Anon says:

    @Frode Aarebrot:

    He asked if anyone was still making new 32-bit processors, not if anyone was still making new x86-compatible 32-bit processors.

  27. Jonathan Wilson says:

    What I want to know is why so many OEMs are shipping PCs with a 64 bit CPU inside yet shipping a 32 bit version of Windows 7 on it…

  28. DWalker says:

    @Troll: "if Windows 8 ships in 64-bit only version"… I predict that Windows 8 *will* ship as 64-bit only.  It may depend on when Windows 8 ships.  My personal guess (with no knowledge) is 3-4 years from now.  Is anyone still making new 32-bit processors and chipsets?  Windows XP will be around with extended support (security patches) until 2014.  

  29. Cheong says:

    @Jonathan Wilson: Because lots of people buy their PC to play games, there's not many PC games targeting x64 yet.

    If there were less rant about unstable 64-bit display drivers, and longer loading time, I suspect there would be lots more people choosing x64 as default.

    Btw, what's the benefit of installing 64-bit version unless your PC has > 4GB RAM installed? Perheps we have to wait until 8GB machine becoming entry class that Microsoft can push a x64 only OS.

  30. GregM says:

    Cheong, simple: getting more than 3GB of address space per process.  Remember, address space does not equal ram.

  31. Cheong says:

    @GregM: I know in x64 your process can have > 3GB address space, but since most application are still 32-bit (or vendor provides both 32/64-bit version), any application that need that much address space in 32-bit system would have found other ways to fit their need anyway. From an average user's perspective, this is really irrelevent.

    [I'll have to remember to use this explanation. "There's no point doing X because everybody has to work around it anyway." Who will tell the Excel and Exchange teams to just use whatever workaround the 32-bit version was using instead of developing a 64-bit version. Oh wait, there was no workaround in the 32-bit version in the first place. -Raymond]
  32. Crescens2k says:

    Cheong:

    There may be doubts until now because the code generation by x64 VC was still inferior to x86 VC due to it being relatively young. But VC2010 is making a huge difference, so when the compiler itself becomes more used then you will start seing major differences.

    But what are the benefits of 64bit systems other than memory? Double the general purpose registers for one, so more data can be kept on processor without having to swap between memory. For lengthy calculations and with an intelligent system this will help make the 64bit version run a lot faster and I have seen it at work already in some places. (One place is the 64bit decoding of 1080p videos is faster due to this). Second you are guaranteed to have SSE2 instructions on the processor, so any code can take advantage of it without explicitly checking. But yes, these do require you building a 64bit version of the application since the 32bit emulation is only there for compatability.

  33. DWalker says:

    Sorry to lead people off the topic, which was cursors…  :-)

  34. Cheong says:

    @Raymond: I'm not saying that there's no point to do x64 development, just saying that because there's so few applications that actually requires x64, there's little reason for average users to use a x64 OS in the meantime unless the user has more than 4GB RAM.

    [You're confusing address space with RAM. -Raymond]
  35. Cheong says:

    I wonder why you don't see the context of my chain of comment before getting that impression. I'm responding to the comment on why there's so many vendors still selling PCs ready for x64 but yet with 32-bit OS, trying to give some reason behind that decision. That's nothing to do with x64 development at all.

    The core part I'm saying is, the average users won't see the difference yet. The additional address space is good, but if you don't have that much RAM, you still have to page them to disk, right? The additional registers on x64 is good, but programs cannot take advantage over them unless the compiler explicitly asked to use them, right?

    But really, is there any common application that average users use require that much RAM/registers that x86 environment cannot satisfy, so only x64 version is available?

    Seems you're interpreting my comments in some way that I don't mean it that way.

    [Address space is used for things other than RAM. And running out of address space is an increasingly common problem in 32-bit code. Even the PC version of Halo ran into it. (More examples in the blog queue.) -Raymond]
  36. Crescens2k says:

    Cheong: x64 compilers use everything they can get ahold of by default. The x64 calling convention even requires the use of some of these registers. If you are curious look at the assembler output of a non trivial program, you will see r8 – r15 in use, there is nothing stopping it from using SSE2 when possible, and it will, there are places in the CRT where it uses SSE2 when available too. So yes the compiler must explicitly use them, but the compiler does explicitly use them.

    But the vendors who only provide x86 versions of Windows now can only see the now. Raymond mentioned Halo had address space issues, and that was a few years back. But the thing is to remember that it happened once and so it can happen again, and again. Once game developers start understanding and being able to use x64 to it's fullest then I wouldn't be surprised to see a jump in the quality of textures and things like that because of the address space available. The problems of a few years ago have also vanished. The original issues was lack of drivers and the like, but over the last few years it has been harder to find a modern supported device without a driver for x64.

    x64 is becoming used more commonly, a couple of years back it was hard to find x64 software. But now there are media players, codecs, web browsers, developer tools, office suites etc which use x64. So only seeing that x64 isn't useful for the average user yet is short sighted and you should look at next year or the year after before making that decision.

    ["32 bits should be enough for anyone." -Raymond]

Comments are closed.

Skip to main content