If I have a modeless dialog box with custom accelerators, which should I call first: IsDialogMessage or TranslateAccelerator

A customer had a modeless dialog box with custom accelerators.

If their window had been a modeless dialog box without custom accelerators, then their message dispatch would be

if (!IsDialogMessage(hdlg, &msg)) {

On the other hand, if their window with accelerators had been a plain window rather than a dialog box, then their message dispatch would be

if (!TranslateAccelerator(hwnd, hacc, &msg)) {

But since they have both, the question arises: Which should I do first, the Is­Dialog­Message or the Translate­Accelerator?

The customer experimented and found that they had to call Translate­Accelerator first:

if (!TranslateAccelerator(hwnd, hacc, &msg) &&
    !IsDialogMessage(hdlg, &msg)) {

If they flipped the order, they found that accelerators were not being translated:

// Code in italics is wrong.
if (!IsDialogMessage(hdlg, &msg) &&
    !TranslateAccelerator(hwnd, hacc, &msg)) {

The customer empirically determined that you have to translate the accelerator first. (Or they could have read my article on the subject of custom accelerators in dialog boxes and seen the correct order.) But why is that the correct order?

The answer goes back to the return values of the two function. The Translate­Accelerator function returns a nonzero value if it recognized the message as an accelerator (and posted a WM_COMMAND message). The Is­Dialog­Message function returns a nonzero value if it recognized the message as a message for the dialog (and dispatched it).

Now look at what happens if you have a message that is both a message for the dialog and an accelerator. For example, focus is on a button control in the dialog box and you press, say, Alt+F2.

Let's say you call Is­Dialog­Message first. The Is­Dialog­Message function says, "Why yes, this message is for the dialog box, so I dispatched it to the button control. Mission accomplished. I'm so awesome!" The Is­Dialog­Message function returns a non-zero value, and the Translate­Accelerator never gets a chance to run.

On the other hand, if you call Translate­Accelerator first, then the Translate­Accelerator function sees the accelerator key and posts the WM_COMMAND function to the dialog window, then it returns a non-zero value to say "Why yes, this message is an accelerator, so I posted a WM_COMMAND message. Mission accomplished. I'm so awesome!" The Translate­Accelerator function returns a non-zero value, and the Is­Dialog­Message never gets a chance to run.

The question of which one to call first is therefore a matter of priority. If the user presses the accelerator key while focus is on a control in the dialog box, which is more important to you: The fact that it is an accelerator, or the fact that it is a message that targets the dialog box? Whichever is more important to you goes first.

But wait, that's not the end of the story. Note that the above code calls Translate­Accelerator unconditionally, which means that the accelerator keys are active even if focus is not on the dialog box at all. For example, focus may be on another window on the same thread (say, the owner of the modeless dialog box). You probably don't want the modeless dialog box stealing accelerators from the owner. To avoid this problem, you need to translate accelerators for your dialog box only if the focus is somewhere in your dialog box.

 if (!((hdlg == msg.hwnd || IsChild(hdlg, msg.hwnd)) &&
       !TranslateAccelerator(hdlg, hacc, &msg)) &&
     !IsDialogMessage(hdlg, &msg)) {

This code should look familiar, since I copied it from my original article.

Comments (14)
  1. wqw says:

    Why does such simple thing as a process message pump quickly turn into an abomination? I might have multiple modeless forms (non-modal dialogs), even from 3rd party dynamicly loaded plugins. How is my process message pump supposed to handle these? Custom registration scheme for current `hdlg` and `hacc`?

    Totally broken design IMHO!

    1. Pietro Gagliardi (andlabs) says:

      I would like to know as well, but I’ve always had the suspicion that this sort of application design is the thing that’s broken from the perspective of the Windows application model, and not the design of IsDialogMessage().

      Or to put it another way, let’s say you have a program like Visual Studio or Photoshop that has lots of possibly-dockable toolbox windows open alongside the main content. Should those individual toolboxes be navigable with the Tab key? If so, how *should* one handle that case? Tracking every WM_ACTIVATE or WM_NCACTIVATE (in the same way as https://blogs.msdn.microsoft.com/oldnewthing/20140521-00/?p=943) to decide what the current dialog window is? GetActiveWindow()?

      A different question that has the same purpose is: Why does IsDialogMessage() take the hdlg parameter in the first place? Couldn’t it just gather the correct dialog from, say, GetAncestor(msg->hwnd, GA_ROOT)? Because as it stands, if I have multiple toplevel windows on the same thread, and a code-modal dialog running (say, IFileDialog::Show()), then tab navigation stops working for the other windows because the system-provided dialog box message loop only calls IsDialogMessage() for that one dialog. Not every message has to have an associated window handle, though, and I’m not sure what IsDialogMessage() would need to watch for from messages without one…

      As for multiple accelerator tables, I can’t think of a good use case for those, apart from either a) having different sets of accelerators for different parts of your program, which I’m pretty sure would just lead to endless confusion, or b) separating a giant accelerator table into multiple smaller ones, one for each part of your program, and calling TranslateAccelerator() multiple times — which seems pointless since the messages that result don’t care which accelerator table is used (so all would have to avoid collisions with accelerator IDs anyway).

      1. Neil says:

        As soon as you invent that time machine I’ll go back in time and get key messages to bubble in the same manner as WM_SETCURSOR so that dialogs can do their own navigation and top-level windows can process their own accelerators without messing up the message loop.

        1. Pietro Gagliardi (andlabs) says:

          Sure, that would avoid the need for IsDialogMessage(), but my point, which I admit probably drowned in the post, was more “Should auxiliary windows have tab navigation in the first place? Or hell, non-dialogs at all?”

        2. If we’re going for a ride in the time machine, then may as well ask for the event to tunnel before bubbling. That way, the dialog box can pick off accelerators before controls see them. E.g., “I want Ctrl+C to copy all the text from the dialog box, even if the user is in a control that normally uses Ctrl+C to mean something else.”

  2. Joshua says:

    And this awful mess is why you don’t want to have more than one message loop per thread if you can help it.

    No fair blaming this design for lack of plugin support. It is inherited from the beginning, from before plugins could be a thing.

  3. There should really be a book called Windows API messages for Dummies.

  4. Myria says:

    There is confusion from the name “IsDialogMessage”, since the function has an effect, rather than merely querying whether the message is a dialog box message.

    1. Tim! says:

      “Why yes, this message is for the dialog box, so I dispatched it to the button control. Mission accomplished. I’m so awesome!”

      I wholeheartedly and sarcastically agree; side effects from a getter-named function are so awesome.

    2. I agree that the function has a terrible name. Better would be something like ProcessDialogMessage.

  5. Yuri Khan says:

    What is Alt+F2?

    1. ghbyrkit says:

      It is an example. you should be able to infer that from how Raymond wrote the text “and you press, say, Alt+F2.”. So he was giving an example of an accelerator sequence (search on the web for it, rather than expect a tutorial on it here…) ‘Alt’ keys generally begin an accelerator key sequence

      1. Nick says:

        99% sure Yuri was asking what Alt+F2 does in a dialog when it’s NOT bound to a custom accelerator. Given that a lot of those types of accelerators are invisible if you don’t know they’re doing something (think Ctrl+C to copy, there’s no visible state change), it seems like a reasonable question.

        1. Yuri Khan says:

          I knew IsDialogMessage handles navigation between controls and Enter and Esc as shortcuts for OK and Cancel, but it was unclear to me why it would decide that Alt+F2 is for the dialog. I was assuming IsDialogMessage is conservative in what it handles.

          However, reading Wine’s source code for IsDialogMessage revealed that it is greedy: its catch-all case dispatches all unrecognized messages whose target is a child of the dialog and returns TRUE.

          (Wine is not Windows but is a reasonable predictor of actual Windows behavior.)

Comments are closed.

Skip to main content