The importance of the FORMAT_MESSAGE_IGNORE_INSERTS flag


You can use the FormatMessage message with the FORMAT_MESSAGE_FROM_SYSTEM flag to indicate that the message number you passed is an error code and that the message should be looked up in the system message table. This is a specific case of the more general case where you are not in control of the message, and when you are not in control of the message, you had better pass the FORMAT_MESSAGE_IGNORE_INSERTS flag.

Let’s look at what happens when you don’t.

#include <windows.h>
#include <stdio.h>
#include <tchar.h>

int __cdecl main(int argc, char **argv)
{
 TCHAR buffer[1024];
 DWORD dwError = ERROR_BAD_EXE_FORMAT;
 DWORD dwFlags = FORMAT_MESSAGE_FROM_SYSTEM;
 DWORD dwResult = FormatMessage(dwFlags, NULL, dwError,
                                0, buffer, 1024, NULL);
 if (dwResult) {
  _tprintf(_T("Message is \"%s\"\n"), buffer);
 } else {
  _tprintf(_T("Failed! Error code %d\n"), GetLastError());
 }
 return 0;
}

If you run this program, you’ll get

Failed! Error code 87

Error 87 is ERROR_INVALID_PARAMETER. What went wrong? Let’s pass the FORMAT_MESSAGE_IGNORE_INSERTS flag to see what the message was. Change the value of dwFlags to

 DWORD dwFlags = FORMAT_MESSAGE_FROM_SYSTEM |
                 FORMAT_MESSAGE_IGNORE_INSERTS;

and run the program again. This time you get

Message is "%1 is not a valid Win32 application.
"

Aha, now we see the problem. The message corresponding to ERROR_BAD_EXE_FORMAT contains an insertion %1. If you don’t pass the FORMAT_MESSAGE_IGNORE_INSERTS flag, the FormatMessage function will insert the first parameter in the argument list (or argument array). But we didn’t pass an argument list, so the function fails.

Actually, we got lucky. If we had passed an argument list or argument array, the function would have inserted the corresponding string, even if the argument list we passed didn’t have a string in the first position.

If you are not in control of the format string, then you must pass FORMAT_MESSAGE_IGNORE_INSERTS to prevent the %1 from causing trouble. If somebody was being particularly evil, they might decide to give you a format string that contains a %9, which is almost certainly more insertions than you provided. The result is a buffer overflow and probably a crash.

This may have been obvious to some people, in the same way that you shouldn’t pass a string outside your control as the format string to the printf function, but I felt it worth mentioning.

Comments (16)
  1. Anthony Wieser says:

    I’ve always wondered who was in control of that string.  It certainly looks bad if a %1 gets back to the user.

    Is there any way to know what should be provided for a given error message, or is it deeply bound up with the implementation inside the system, and hence inpenetrable.

    The fact that you say “you had better pass the FORMAT_MESSAGE_IGNORE_INSERTS flag” implies there is no way to know what to pass, which is a shame.

    [It bugs me too that system error messages contain %1 insertions that you just have to “know” on a case-by-case basis. -Raymond]
  2. Tom says:

    Kind-of on topic: Do you know why Visual Studio does not allow you to insert a message resource from the Add Resource menu?  I know I can add custom build steps to get it done, but the fact the feature is not present always made me wonder if using message resources (as opposed to a string table or some other localized error message mechanism) was the proper way of handling error strings.

  3. Matthew says:

    Damn, hadn’t realized this.

    One guilty codebase now fixed — thanks!

  4. Anthony Wieser says:

    What worries me more Raymond, is that though it works today if I "know" what it does, there’s not necessarily a contract that says it’ll stay that way.

  5. b says:

    Thanks for this article.  I had been misusing FormatMessage() not realizing that some system error codes had insertions.

  6. meh says:

    Yes, definitely worth mentioning. Thanks, Raymond. I did suspect that something like this may happen, but I wasn’t truly convinced. Fortunately I figured… better safe than sorry – and used both flags. Now we all know for sure.

  7. Peter says:

    Like all readers of your blog (I hope), I just checked my company’s code.   I found that (ahem) I had already done this check.  Not only that, but I found that:

    // of the 2906 messages in winerror.h,

    //   2874 messages take no parameters

    //      26 messages take 1 parameter

    //      5 messages take 2 parameters

    //      1 message takes 3 parameters

  8. f0dder says:

    Thanks, that’s indeed something you could get bitten by. Fortunately, I’ve only ever used FormatMessage by copy-pasting the example snippet from MSDN, which thankfully has IGNORE_INSERTS :)

  9. GreaseMonkey says:

    Reminds me of AOE 2: "Click here to select this %s."

  10. Worf says:

    Is it just me, or does FormatMessage end up being a huge PITA function to actually use? Nevermind this flag and the random number of insertions (more errors can be added as time goes on, so you can’t ever predict the error down the road…), but just trying to return something useful to the user.

    Especially when a decent C library has strerror() the takes one argument – the error number – and returns a static pointer to localized error messages. Which can then be used directly via %s and friends inline… a nice simple to use function.

    At least, I’ve given up using FormatMessage…

  11. Neil says:

    It looks as if this faulty code snippet was lifted from net.exe – try "net helpmsg 193" – although ironically message 3871 which does include an insert has had to be special-cased!

  12. Ruslan says:

    I know you moderate comments, so this is an appropriate way to respond you regarding your previous post, without irrelevant stuff appearing in this one.

    Agree, I wrote bullshit myself, so there’s no arguing here. My apologies for being not polite. But there’s I thought I’d convey.

    Putin’s done a lot for Russia. Probably, as much as the harm Bush has done for the US (but with a positive sign, and not in absolute numbers, because the US economy is bigger). So, no doubt, Putin remaining in power is good for Russia. Of course, not in a such stupid way as changing or breaching the constitution, as that’d be not good for Russia. So all critics about Putin remaining in power is wrong in two ways: wrong in principle, and wrong in that he’ll try to break the law.

    In case you wonder: I’m not a Putin fan. There’re things that have become worse during his rule; and there’re things that haven’t improved. (that’s why I immigrated) But it’s nothing compared to the good things he’s done. You’d not expect a human to be 100% flawless, right?

    And, third, you blame Putin and write what you write (which is your 100% right, of course) because you are a victim of Western propaganda. You Americans believe in freedom of your mass media so much that you apparently don’t question its objectiveness. It’s good, indeed, and I’ve been following it for quite a while. So what I also noticed is that it’s not free from engagement and sometimes serves as a media for distributing disinformation, even to Americans themselves. It was so about Yugoslavia, Iraq, etc. In relation to Putin, the position of the Western mass media is always negative. Yet your investment banks (I work in one of them) increase their investments into Russia by billions every half year. Would they do so if they thought situation in Russia deteriorates?

    [If you want to respond to me personally, then send me email. Don’t hijack an unrelated thread. And I think you missed the part of my message where I explicitly pointed out that I don’t have a position on whether it was a good idea that Putin stay in power (I don’t know enough as you more than amply rant on — not sure why you’re ranting about something I already admitted); I was merely being amused at all the effort at figuring out how to do it. Normally I would delete this comment but then it would confirm your incorrect assumption that doing so was acceptable behavior. -Raymond]
  13. Wow. Russian political spam.

  14. pingpong says:

    @Ruslan:

    You’re not a Putin fan, but you decide to defend him in the technical topic. Is there a real Putin fan standing nearby, holding a gun next to your head?

  15. DriverDude says:

    It seems to me that FormatMessage was intended for locale-specific message lookup, more like gettext rather than the simplistic strerror.

    As with anything that complicated, perhaps over time people forgot how FormatMessage is used. Insertion strings are great if the caller knows what to insert; caller doesn’t have to be concered where to insert it. Not so great as a generic error message lookup facility.

    For example, there are plenty of erorr messages of the form "Semaphore period expired" or "File not found", instead of "%1 could not be found"  

    Seems to me the person who added ERROR_BAD_EXE_FORMAT forgot that FormatText is sometimes used to generically translate error numbers. Or maybe that guideline was never published.

    Thanks for the lesson in software engineering, Raymond.

  16. Anonymous Coward says:

    After reading this article (thanks for posting it btw – I discovered that we did indeed have this very same bug in our software as well), it seems like FORMAT_MESSAGE_IGNORE_INSERTS should be implied by FORMAT_MESSAGE_FROM_SYSTEM, and you should have to explicitly pass in another flag (maybe FORMAT_MESSAGE_ALLOW_INSERTS or something similar) to get the inserts expansion behavior. Of course I realize that it’s too late for FormatMesssage – that ship has already sailed – but hopefully future APIs will not repeat the same design.

Comments are closed.