Using the wrong HINSTANCE in RegisterClass is like identity theft


Last year, I left you with a teaser for a problem that resulted in the CResource­Exception being thrown.

Studying the function that threw the exception revealed that it was thrown due to a failed call to Register­Class. And studying the parameters that were passed to Register­Class revealed that HINSTANCE parameter did not match the DLL. Instead of being the instance handle of the DLL, it was the instance handle of the host application.

Okay, now let's apply what we learned a few years ago about the significance of the HINSTANCE parameter passed to the Register­Class function. By passing the HINSTANCE of the host application, the class was registered against the namespace of the host rather than the namespace of the DLL. It's like signing up for a credit card using somebody else's name or checking a book out of the library with somebody else's library card.

In this case, the module in question was a plug-in. It tried to register a class called, say, My­Class, and instead of registering against itself, it registered against the host application. Fortunately, the host application didn't have a class called My­Class, so the incorrect registration didn't cause a conflict. The book got checked out to the wrong person, but as far as the library can tell, nothing has gone wrong. It merely looks like the host application checked out a book.

So why did the call to Register­Class fail? Because some other plug-in made the same mistake. Plug-in B also registered its class against the host application, and by an amazing coincidence, its class was also called My­Class. (If you look at how MFC auto-generates class names, you can see that this name collision can happen quite easily.) If both plug-ins had registered their classes properly, there would have been no problem, because each class would have been registered against their respective DLLs, and no conflict would have arisen. But instead, two wrongs make a wronger, and since both plug-ins incorrectly registered their classes against the host, the first plug-in to register succeeds, and the second one crashes.

(One might argue that this is another special case of What if two programs did this?)

Both plug-ins tried to sign up for a credit card in the name of the host application. The first one got the card, and the second one was informed by the credit card company, "Your application was denied because you already have a card from us."

Why did these plug-ins register against the host application instead of against their own library cards? I don't know for sure, but my guess is that it was due to ignorance.¹ When reading the documentation, they found that they needed to fill in the hInstance member of the WNDCLASS structure. Gosh, where do I get an HINSTANCE from? Oh wait, I found a function that returns an HINSTANCE: Get­Module­Handle. And hey look, if I pass NULL, a valid HINSTANCE comes out. I'll just set wndclass.hInstance = Get­Module­Handle(NULL); and try it. Hey, look, it works!

Footnote

¹ This reminds me of a story that took place at an administrative hearing. The government agency representative presented as evidence that the other party admitted in a telephone conversation to being ignorant of the applicable regulations.

The other party angrily interrupted.

"I'm not ignorant! I simply didn't know what the rules were."

The judge patiently explained, "That's what the word ignorant means."

Comments (19)
  1. Tringi says:

    It is far too common to assume that GetModuleHandle(NULL) is equivalent to something like GetCurrentModuleHandle (which is not that difficult to write yourself, google for it).

  2. Joshua says:

    I'm in the habit of saving my HINSTANCE in DLLMain to avoid this kind of problem.

  3. jader3rd says:

    My mom still believes that ignorant means going out of your way to be mean. We've tried to tell her that that's not what it means, but she claims that's how everyone else (ie, everyone but her spouse and children) uses the word.

  4. David Walker says:

    "Two wrongs make a wronger".  How true.  I'll have to remember that phrase.

  5. Adam Rosenfield says:

    Ah, the politician's fallacy:  "Something must be done.  This is something.  Therefore we must do this."  

  6. Disparato says:

    How can I get the correct HINSTANCE if my code lives in a static library that is used both for building EXEs and DLLs? Passing in an HINSTANCE is not an option.

  7. Moi says:

    @Disparato

    Raymond has covered this before. Search for __ImageBase.

    blogs.msdn.com/…/247180.aspx

  8. Jeremy says:

    If a person knows or finds out that a valid library card is required to check out a book, I cannot believe that a "reasonable person" (en.wikipedia.org/…/Reasonable_person) would think that using someone else's library card without their knowledge would be a viable solution.  I think in this case, ignorance is neglegence.  We need some sort of protection against neglegent and reckless programmers!

    en.wikipedia.org/…/Mens_rea

  9. In Redmond, library cards require a PIN for self-checkout.  Not sure how your identity is verified when you check out at the librarian's desk.

  10. Mark (The other Mark) says:

    Does anyone have a reference to the story in the footnote? That sounds interesting, but right now my searches mostly point to blogs copying this one.

    [Reference: Personal communication. -Raymond]
  11. Myria says:

    If you don't like the __ImageBase trick, or you're using a different compiler like GCC, you can also do this, if you can require XP or later:

    static const wchar_t s_empty[] = L"";

    if (!GetModuleHandleExW(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS | GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT, s_empty, &wndclass.hInstance))

    {

     // error

    }

    s_empty will be within your module.

  12. MJP says:

    @Ray: Try translating this article to german with the MS Translator…

  13. skSdnW says:

    @Myria: If you check the link posted by Moi you will see the GetModuleHandleEx trick and the older VirtualQuery variant…

  14. hagenp says:

    "When reading the documentation, they found that they needed to fill in the hInstance member of the WNDCLASS structure."…but no additional information or example that explains _why_ and how to use the function. Hence documentation-induced errors are created.

    [Its purpose is explained in the overview article. And the "Reference" section of the WNDCLASS documentation links to all the overview articles. -Raymond]
  15. henke37 says:

    It is documented, it just so happens that the documentation assumes that the reader knows what an instance is in the first place.

  16. 640k says:

    API is not helping. Should be designed better.

    [It's not enough to say that something is bad; you have to say what would be better. -Raymond]
  17. 640k says:

    I don't get paid for designing windows api, i'm only the sucker who use them. Please google (or bing!) on "pit of success".

    [You don't have to design it, just say wat would be better — i.e., how you would prefer to use it. "I don't want to have to pass an instance handle parameter. The system should just figure it out somehow." (Answer: "Somehow" involves guessing.) -Raymond]
  18. Jules says:

    @640k: to expand on Raymond's point a little, a guess is likely to be wrong in at least some cases, because there are at least three entirely different use cases for a DLL registering a window class: (1) the DLL is going to create windows itself, (2) the DLL is providing the behaviour for windows that the main application (or some other DLL also loaded by the application) is going to create, or (3) the DLL is simply registering the class on behalf of a caller, and is in no way involved with any interpretation of what the parameters mean.  So either you expand the API to account for all these possibilities (possibly along with others I haven't thought of), or you require the DLL to provide details of a namespace of some kind.  Now, the instance handle is probably the best bet for a namespace, because it is the only thing that changes if the same code is included twice in two different DLLs (e.g. because two different parts of an application are using two different versions of the same DLL).

    The only other possibilityies increase the complexity of creating windows (because they prevent you from using constant class names to identify the type of window you're creating; you'd need to use either some kind of window class handle or a pointer to a WNDCLASS structure or similar), which is hardly beneficial.  All to save people having to store a handle in a global variable at startup time?  Absurd.

    Now, don't get me wrong, the Windows API is a long way from flawless, but this doesn't seem too bad to me.  No other possible choices look all that much better from where I'm sitting.

    [Aside — I don't suppose anyone knows why I always have to enter my posts twice whenever I'm trying to post here, do you?  It's quite predictable.  Using Firefox 3.6 on XP, noscript enabled but allowing scripts to run on msdn.com and microsoft.com]

  19. 640k says:

    If there 3 completely different use cases for a single API function, that function should be spit into three different functions.

    But if doesn't help, the whole architecture should be changed to not require the "api user" to guess.

Comments are closed.