Those who do not understand the dialog manager are doomed to reimplement it, badly


A customer wanted to alter the behavior of a multi-line edit control so that it did not treat a press of the Tab key as a request to insert a tab character but rather treated it as a normal dialog navigation key. The approach the customer took was to subclass the edit control and intercept the Tab key:

LRESULT CALLBACK SubclassWndProc(
    HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
  switch (uMsg) {
  case WM_KEYDOWN:
    if (wParam == VK_TAB) {
      // Pressed the TAB key - tab to next control
      SetFocus(GetNextDlgTabItem(
                     GetParent(hwnd), hwnd, FALSE));
      return 0; // message handled
    }
  }
  return CallWindowProc(...);
}

There are many things wrong with this approach. You can spend quite a lot of time nitpicking the little details, how this code fails to set focus in a dialog box properly, how it fails to take nested dialogs into account, how it fails to handle the Shift+Tab navigation key, how it blatantly assumes that the control is part of a dialog box in the first place! But all of these little details are missing the big picture: Instead of fighting against the dialog manager and reimplementing all the parts we want to keep and ignoring the parts we want to skip, we should be working with the dialog manager and expressing our intentions in the manner the dialog manager expects.

It's the difference between ordering a hamburger without pickles and ordering a hamburger with pickles, and then carefully picking the pickles off the burger when you get it.

In this case, we want to prevent the edit control from saying "Give me the Tab key." We saw last time that this is done either by (1) setting the DLGC_WANTTAB dialog code or by (2) responding with DLGC_WANTMESSAGE when given a Tab key message. Therefore, to tell the dialog manager to not to treat the tab key specially, just turn off those two behaviors.

LRESULT CALLBACK SubclassWndProc(
    HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
  LRESULT lres;
  switch (uMsg) {
  case WM_GETDLGCODE:
    lres = CallWindowProc(...);
    lres &= ~DLGC_WANTTAB;
    if (lParam &&
        ((MSG *)lParam)->message == WM_KEYDOWN &&
        ((MSG *)lParam)->wParam == VK_TAB) {
      lres &= ~DLGC_WANTMESSAGE;
    }
    return lres;
  }
  return CallWindowProc(...);
}

After asking the original control what behavior it thinks it wants, we turn off the DLGC_WANTTAB flag; this takes care of part (1). Next, we check whether the message is a press of the Tab key. If so, then we turn the DLGC_WANTMESSAGE flag off; this takes care of part (2).

This is certainly less code than would need to have been written to address all of the little concerns noted earlier, and it does it by completely sidestepping the task of trying to emulate the dialog manager and instead just cooperating with the dialog manager to get the behavior you want. This principle of "If you know how a system is meant to work, you can work with it rather than against it, and everybody will be happier for it" is something I've been trying to convey through this web site and my book. Knowing how something was intended to be used allows you to be a more effective programmer.

Exercise: Why didn't we just write this instead?

LRESULT CALLBACK SubclassWndProc(
    HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
  LRESULT lres;
  switch (uMsg) {
  case WM_GETDLGCODE:
    lres = CallWindowProc(...);
    lres &= ~DLGC_WANTTAB;
    if (wParam == VK_TAB) {
      lres &= ~DLGC_WANTMESSAGE;
    }
    return lres;
  }
  return CallWindowProc(...);
}
Comments (23)
  1. Anonymous says:

    I’m going to shoot and say ‘WM_KEYUP’ will also trip the exercise code and you will get two tab effects for every press of the key

  2. Anonymous says:

    Concerning your exercice, I’d say that wParam meaning is tied to the message field value, therefore it would work only if VK_TAB is an unique value for the wParam field.

    Maybe there is another message which defines a  constant whose value equals to VK_TAB.

  3. Anonymous says:

    To quote Wrath of Khan: "You’ve got to learn WHY things work on a Starship."

  4. Anonymous says:

    wParam isn’t used for WM_GETDLGCODE. I’m guessing that some people try the "right" way first (subclassing for WM_GETDLGCODE, checking wParam for the key) and then give up when it doesn’t work ("Stupid Windows…") and they never get around to looking up what they’re actually supposed to do. (check the wParam through the lParam MSG pointer)

    Depressing, but entirely possible.

  5. Mike Dimmick says:

    Re the Exercise: Perhaps it’s because wParam is not defined for a WM_GETDLGCODE message. We need to know what the original message’s wParam was. If the original message is anything other than a WM_KEYDOWN, wParam could have a different meaning entirely. And you have to test that it’s really a message, because the dialog manager sends WM_GETDLGCODE when it’s about to set focus to a control, to see whether the control sets DLGC_HASSETSEL and therefore the dialog manager should send an EM_SETSEL message, or the control sets DLGC_UNDEFPUSHBUTTON and it should make the button the default button.

    Alternatively the application may be moving the focus itself by sending WM_NEXTDLGCTL, where wParam is the ID of the control you’re moving to (I’m not sure if the dialog manager passes this message along to the control). You should use this rather than SetFocus to ensure that the control gets sent EM_SETSEL (if an edit control, or otherwise returning DLGC_HASSETSEL) or, if a button (or DLGC_UNDEFPUSHBUTTON), that it becomes the default button – in other words, exactly the same as if the user had tabbed to the control.

    In the example, you don’t need to check for WM_CHAR or WM_KEYUP because if it works, the focus has moved to the next control. Having said that, if for some reason there are no other active controls on the dialog, no focus change will occur. You should check that this is correct (personally I would handle Tab being pressed in WM_CHAR rather than WM_KEYDOWN).

  6. Anonymous says:

    Because you need the wparam of the WM_KEYDOWN message, not the WM_GETDLGCODE message.

  7. Anonymous says:

    I have something similar in my program only with a RichEdit control and the escape and enter keys. Your post made me wonder why it was there in the first place so I dug in a little. It turns out the RichEdit control eats every key including VK_ENTER and VK_ESCAPE even though ES_WANTRETURN isn’t used. To make up for it, it sends WM_CLOSE in response to the escape key, WM_NEXTDLGCTL in response to the tab key and WM_KEYDOWN in response to the the enter key.

    But where does it send it to? Its parent dialog. In my case, that dialog is an internal (DS_CONTROL) dialog so WM_CLOSE and WM_KEYDOWN are sent to the wrong place.

    I’d call that a good example to what can go wrong when you implement the dialog manager on your own.

  8. Anonymous says:

    One problem with VB4/5/6 and writing ActiveX controls for it (in VC++) was the thunder forms completely ignored the dialog box flags for keystrokes. (IIRC the controls worked fine in MFC dialog boxes and IE)

    Which meant that you had to use PreTranslateMessage to grab the arrow keys if you wanted them (or at least that was the advice I got from one of the mailing lists at the time). I suspect there may be an ‘nice’ ActiveX container method of grabbing the arrow keys but I couldn’t find it at the time…

  9. Mike Dimmick says:

    @Erzengel: if you have the code, not the spec, you end up coding to the implied spec that you have intuited from the code, not the actual spec. This causes compatibility problems when the next version of the OS implements the real spec differently.

    READ THE DOCUMENTATION.

  10. Anonymous says:

    Peter: You also can’t seem to hook CreateWindow()d windows in with the VB forms tab order and focus handling code.

    I expect there is a way but I couldn’t find it last I looked and its certainly not documented.

  11. Anonymous says:

    Without source code for reference, Windows programming is an endless  stream of folklore. This blog is an entertaining and bountiful part of the stream, but it is still just folklore.

    How bad can it get? In 1990, I wrote the following:

    case WM_MOVE: MoveWindow(hWnd, …

    It seemed to be what the documentation called for.

  12. Anonymous says:

    FYI, WM_GETDLGCODE documentation is at http://msdn2.microsoft.com/en-us/library/aa453861.aspx . You’ll never guess how I found it–I searched for "WM_GETDLGCODE" on Google/Live Search.

  13. Anonymous says:

    "This principle of "If you know how a system is meant to work, you can work with it rather than against it, and everybody will be happier for it" is something I’ve been trying to convey through this web site"

    The problem is that we don’t really get all that much information as to how windows is supposed to work (You’re rectifying that, thank you). That’s why so many open-sourcers demand windows source code. Not so much so they can copy it, but know how to program *for* it better. You have no idea how many times I’ve looked in documentation, found nothing of help, and finally spelunked into the source code of a library and go "Ooooh, *that’s* why!"

    (of course, I understand why Microsoft doesn’t release the source code. The code I write is about as far away from open source as possible, even further than Microsoft… But we don’t have "clients" that need to access our code.)

  14. Anonymous says:

    I’ve come across many types of documentation… I’m tempted to rewrite some of it after I figure out how it’s actually supposed to work.  Just about the best documentation I’ve seen is for PHP, but even then it’s not perfect.  (There’s questions and sometimes unhighlighed "Oops!" cases in the comments.)

    I do agree that understanding "Why" something is supposed to work is just as important as knowing that it works.  That’s why you go over so much theory in college.

  15. Anonymous says:

    <<The problem is that we don’t really get all that much information as to how windows is supposed to work (You’re rectifying that, thank you). That’s why so many open-sourcers demand windows source code. Not so much so they can copy it, but know how to program *for* it better. You have no idea how many times I’ve looked in documentation, found nothing of help, and finally spelunked into the source code of a library and go "Ooooh, *that’s* why!">>

    Definitely.  When doing managed programming, I frequently fire up Reflector and go looking at the implementations of framework classes to figure out why weird things are happening.  (And occasionally to curse at the designers for making something I want to extend private or internal, but that’s a different story.)

    And I’m not trying to pick on MS, here.  MS’s documentation, while sometimes inadequate, is still a *lot* better than most third-party documentation.  Nobody has ever written perfect documentation (nor I suspect ever will) — at least not that I’ve seen.

  16. Anonymous says:

    Could you also explain how to handle the Enter key in Edit control of Dialog?

    I want my Edit control to send message like WM_COMMAND(idCtrl) on Enter key. But Enter key makes dialog to execute default button command.

  17. Anonymous says:

    While it’s a nice, pat answer to say things like, “FYI, WM_GETDLGCODE documentation is at http://msdn2.microsoft.com/en-us/library/aa453861.aspx . You’ll never guess how I found it–I searched for “WM_GETDLGCODE” on Google/Live Search,” it fails to take into account the fact that there is no easy way to even know WHAT to look for in in the first place.  

    Given the thousands of Windows messages, trying to find the magic message that does what you need is often an incredibly difficult first step to overcome.

    As someone mentioned earlier, Windows programming is a stream of folklore.  It has always felt to me that Windows messages have been added to the OS over time within MS by people who needed to get a job accomplished for a project at MS.  Orthogonal is NOT the word I would use to describe the messages in Windows.

    [Gosh, I want to customize the dialog box keyboard interface. Maybe I should read “Dialog box keyboard interface,” linked to from “Dialog Box Overview,” linked to from the “See Also” section of the documentation for every dialog box-related function. Nah, that’s be crazy. -Raymond]
  18. Anonymous says:

    That MSDN page linked by Affine is really poor.  But again it’s the damn page for Windows CE and not the one for Win32, which has a link back to the Dialog Box Overview.  This is what people do here at work too, they do a google search instead of looking at the online help in VC++!

    This has nothing to do with Raymond, of course, but I wish they would 1) get rid of the damn Window CE help files, which is a confusing duplicate of the Win32 help with missing information (People who work on CE should have  to download and install it!) 2) Make VC++’s help file FAST so that it’s not faster to go on the web!  The help was great in VC++ 6.0 IMHO, if you did not install MSDN, and while the bad formatting is slooowly getting fixed, the speed and size is still not what it should be.  I still run VC++ 6.0 next to VC++ 2005 just to access Find-In-File and online help.

  19. Anonymous says:

    Then people give up on Win32 and use MFC. What is worse they use even that incorrectly. Even if they don’t, next version of MFC usually breaks everything that worked before.

  20. Anonymous says:

    "[Gosh, I want to customize the dialog box keyboard interface. Maybe I should read "Dialog box keyboard interface," linked to from "Dialog Box Overview," linked to from the "See Also" section of the documentation for every dialog box-related function. Nah, that’s be crazy. -Raymond]"

    Suppose I want to customize a mouse message. The above extremely reasonable approach is … less helpful. You just gotta know (IT’S FOLKLORE).

  21. Anonymous says:

    "Make VC++’s help file FAST so that it’s not faster to go on the web!"

    I think the reason the new help is so slow is that it is silently going on the web itself before showing you the pages you wanted :-)

    But yes – simple old help was much better, as was the old find dialog. Oh well, newer is better remember.

  22. Anonymous says:

    Exercise: it’s not even documented what lParam does (in MSDN), so how could we even know ?

    [Odd. My version of MSDN documents both wParam and lParam. -Raymond]
  23. Anonymous says:

    [Odd. My version of MSDN documents both wParam and lParam. -Raymond]

    VC++ 6.0’s help isn’t fast enough for me; by default I use the last .hlp format file that Microsoft produced, but unfortunately that doesn’t document wParam or lParam either.

    But then, even after reading MSDN I don’t understand the exercise.

    (Feel free to excise some or all of this rant; I’ve been on holiday so I can’t comment on some of the earlier posts. I was wondering why Raymond used memmove instead of memcpy. I also prefer the XP start menu because it uses the correct disconnect icon (2003 Server gives you Shut Down instead). And to the guy who can’t wait 50ms for flyout menus, just click on the damn thing, ok? I actually prefer clicking on menus, but what kills me is that the flyout menus take so long to appear on our old servers that when I try to click on the menu to open it it’s already started opening so I end up clicking on Terminal Services Manager.)

Comments are closed.