The forgotten common controls: The ShowHideMenuCtl function


The ShowHideMenuCtl function is one of those functions everybody tries to pretend doesn't exist. You thought MenuHelp was bad; ShowHideMenuCtl is even worse.

The idea behind ShowHideMenuCtl was that you had a window with a menu as well as controls, and some of the menu items were checkable, indicating whether the corresponding control should be shown. For example, on your View menu you might have options named Toolbar or Status Bar. If the user checks Toolbar, then the toolbar is shown in the main window; if the user unchecks Toolbar, then the toolbar is hidden.

The parameters to the ShowHideMenuCtl function are a window (the window on which you want to operate), a menu identifier (the menu item you wish to toggle), and a mysterious array of integers. Everything hangs on that mysterious array of integers, which takes the following form (expressed in pseudo-C):

struct MENUCONTROLINTS {
 int idMenu;
 int idControl;
};

struct SHOWHIDEMENUCONTROLINTS {
 int idMainMenu;
 HMENU hmenuMain;
 MENUCONTROLINTS rgwMenuControl[];
};

The MENUCONTROLINTS structure is easier to describe. It merely establishes the correspondence between a menu item and the control that will be shown or hidden. (Exercise: Why do we need two integers? Why can't we just give the menu item and the control the same ID?) The array of MENUCONTROLINTS structures is terminated by a pair whose idMenu is zero.

The tricky bit is the first two entries, idMainMenu and hmenuMain. The hmenuMain is the handle to the main menu for the window, and the idMainMenu is the item on the menu corresponding to the "Hide menu" entry on the main menu. (That's why hmenuMain need to be passed explicitly. We would normally use GetMenu(hwnd) to get the handle to the main menu, but if we've removed it, then GetMenu(hwnd) will return NULL.) If you don't want to have a "Hide menu" option, you can just put a dummy value in the idMainMenu slot that doesn't correspond to any menu item. (The value -1 is probably most convenient for this. Don't use zero since it terminates the list!)

When you call the ShowHideMenuCtl function, it searches for the menu item you specified and toggles the check mark next to that item. What happens next depends on what type of item was found.

  • If the item is idMainMenu, then the main menu is attached to or removed from the window (by using the SetMenu function, of course), corresponding to the check box.

  • If the item is idMenu, then the corresponding control is shown or hidden (by using the ShowWindow function, of course), corresponding to the check box.

That's all there is to it. The rest is up to you. For example, when a control is shown or hidden, it's still up to your program to relayout the visible controls to account for the new window visibility state. For example, if the user shows the toolbar, then the other controls need to move out of the way to make room for the toolbar. The ShowHideMenuCtl function can't do this for you since it has no idea what your window layout is.

Let's put this information into practice. Start with our scratch program and make the following changes;

HMENU g_hmenuMain;
INT rgiMenu[] = {
    100, 0,
    101, 200,
    0, 0,
};

BOOL
OnCreate(HWND hwnd, LPCREATESTRUCT lpcs)
{
    /* We'll talk about this line more later */
    rgiMenu[1] = (INT)GetMenu(hwnd);

    CreateWindow(TEXT("Button"), TEXT("Sample"),
                 WS_CHILD | BS_PUSHBUTTON, 0, 0, 100, 100,
                 hwnd, IntToPtr_(HMENU, 200), g_hinst, 0);
    return TRUE;
}

void
OnDestroy(HWND hwnd)
{
    if (!GetMenu(hwnd))
        DestroyMenu(IntToPtr_(HMENU, rgiMenu[1]));
    PostQuitMessage(0);
}


void OnCommand(HWND hwnd, int id, HWND hwndCtl, UINT codeNotify)
{
    switch (id) {
    case 100:
    case 101: ShowHideMenuCtl(hwnd, id, rgiMenu); break;
    }
}

HANDLE_MSG(hwnd, WM_COMMAND, OnCommand);

BOOL
InitApp(void)
{
    ....
    wc.lpszMenuName = MAKEINTRESOURCE(1);
    ....
}

/* add to resource file */
1 MENU PRELOAD
BEGIN
    POPUP "&View"
    BEGIN
        MENUITEM "&Menu Bar", 100, CHECKED
        MENUITEM "&Button", 101
    END
END

Most of the changes are just setting up. We attach a menu to our window with two options, one to hide and show the menu bar, and one to hide and show our custom button. Since our window starts out with the menu bar visible and the button hidden, our menu template checks the "Menu Bar" item but not the "Button" one.

The OnCreate function finishes setting up up the rgiMenu array by putting the main menu's handle into index 1 in the array of integers, which corresponds to hmenuMain in our pseudo-structure. The OnDestroy function destroys the menu if it isn't attached to the window, since menus attached to a window are destroyed automatically when the window is destroyed. The magic happens in the OnCommand handler. If the user picked one of our two menu items, then we ask ShowHideMenuCtl to hide and show the button or menu.

The tricky bit is setting up our rgiMenu. Let's break down those integers.

100 Menu identifier for hiding and showing the menu bar
0 Placeholder (receives main menu handle in OnCreate handler)
101 Menu identifier for hiding and showing the menu bar
200 Control ID for the button that is shown and hidden (passed to the CreateWindow function)
0, 0 List terminator

When you run this program, you can use the "Button" menu option to hide and show the button, and you can use the "Menu Bar" menu option to hide and show the window's main menu. Erm, no wait, you can't use it to show the main menu, because the main menu is hidden! Naturally, if your program uses the ability to hide the main menu, you need to provide some alternate mechanism for bringing the main menu back, say via a hotkey or by adding an option to the System menu.

Okay, now back to that line in the OnCreate function that I promised to talk about. If you have been paying attention, alarm bells should have gone off in your head at the line rgiMenu[1] = (INT)GetMenu(hwnd); because we are casting an HMENU to an integer. On 64-bit machines, a HMENU is a 64-bit value, but integers are only 32-bit. This cast truncates the handle value and consequently is not 64-bit safe. Since the ShowHideMenuCtl function requires an array of integers, you're stuck. You can't shove a 64-bit menu handle into a 32-bit integer. The ShowHideMenuCtl function is fundamentally flawed; it is not 64-bit compatible.

Fortunately, nobody uses the ShowHideMenuCtl function anyway. Its functionality is so simple, most programs have already written a function that does roughly the same thing, and since you have to write the layout code anyway, the ShowHideMenuCtl function doesn't really save you very much effort anyway. Like MenuHelp, the function is entirely vestigial and isn't something you should be tempted to use in any modern program. It's a leftover from the days of 16-bit Windows.

Why does such a confusing function exist at all? Well, the shell team thought they were doing you a favor by providing this function back in the 16-bit days. This was originally an internal function used by (I think it was) File Manager, but since it solved a more general problem, the function was exported and documented. In the intervening years, the problem it addressed has been solved in other ways, and the introduction of 64-bit Windows rendered the original solution unworkable anyway, but the function and the code behind it must still linger in the system for backwards compatibility purposes.

The shell team learned its lesson. It no longer exports every little helper function and custom control for third parties to use. If a future version of Windows no longer needs the helper function, or if a redesign of Windows Explorer removes the need for that custom control (or worse, changes the behavior of that custom control), the shell would still have to carry all the code around for the unused function or control because a function, once documented, becomes a continuing support burden.

Comments (17)
  1. Ulric says:

    Useless… and that integer array is simply dangerously designed.  I’ve found also GetEffectiveClientRect in the doc while looking this one up.  Baffling API design style, even for early 1990s.   That one is passing two integer for each control, just to be able to know where the array ends! Or perhaps to match some kind of internal data structure while it was written.  Doesn’t like this was meant to be published. :)

  2. Working on the Visual Studio Tool Platform team, we do a lot of work on making VS extensible and providing

  3. Neil says:

    Sorry for posting here, but new comments to that post are disabled, although it’s still linked to prominently…

    And speaking of disabling comments to posts, do you know if they’ve started timing out earlier? I always thought I used to be able to comment on all the posts on the front page.

    [As explained in the suggestion box, the suggestion box backlog extends into 2010. If your suggestion is that awesome, then I’m sure you’ll remember to submit it when the box reopens in two years. -Raymond]
  4. Koro says:

    “It no longer exports every little helper function and custom control for third parties to use.”

    Then every developper will develop their own sligthly different control that mimics the one the Shell uses (like the preview pane in XP folders), so you end up with another problem instead.

    For example, why isn’t DirectUI documented? It’s used in a lots of places (LogonUI, MSN Messenger, etc…). Instead we’re stuck with WPF which only works with .NET…

    [Do people complain to Lotus, “You need to document your internal UI library so I can write my program that mimics Lotus 1-2-3”? -Raymond]
  5. Mike says:

    “Do people complain to Lotus”…

    Isn’t the mentioned “MSN Messenger” simply an application? If so, it better not make use of non-documented functions in Windows, as I’m sure all know.

    Or is “MSN Messenger” now considered by Microsoft to be a part of the operating system?

    [As far as I’m aware, the version of the DirectUI library in MSN Messenger does not use any undocumented functions in Windows. I don’t know why I’m bothering to respond to this; it’s a lose-lose for me. -Raymond]
  6. Triangle says:

    So just how many programs would break if this function was deleted ?

    (Just so I can put things in perspective)

    [It’s impossible to say. I could write a program tonight, and that would increment the number by 1. -Raymond]
  7. CGomez says:

    Yeah, this brings up a good example of where law and software engineering meet.

    Before software research ever brought up contract programming, the Windows API was already a contract.  It became one just by virtue of being born and then by Windows growth and popularity.

    So it is prudent for these API developers to not just export everything (which adds it to the contract), but to release things in different ways.

    Sometimes, you’ll have no choice but to expose some functionality.

    Sometimes, you aren’t sure if people need this functionality.  Maybe you try it yourself in a few projects (like MSN Messenger or Office) and see if it’s working out.  Maybe there is a need for the functionality to be exported, but it could be improved upon first, before casting it in the everlasting mold of backward compatibility.

    The problem is some stupid people went crying to the DOJ and convinced non-engineers that MSFT perfectly designs everything on the first try and if they don’t export and document everything, then they are a ruthless monopoly.

    It’s a no-win situation for MSFT.  I’m impressed with the quality of people who work there under those conditions.

  8. Bikedude says:

    I remember reading that window handles just use 16-bit under 32-bit Windows in order to better support Win16 apps. 64-bit Windows dropped support for Win16 apps, so now window handles use 32-bit in order to support Win32 apps better. (a huge overkill given a window manager that starts sweating bullets at 20000 windows)

    Which brings me to my question: Is a 64-bit HMENU value really 64-bit? (Given 64-bit Windows — I understand Win128 will probably change things)

    Rune

  9. Bikedude says:

    Oh, I forgot to add (I’m under a contract): Buy m0re v1agr4!

    Rune

  10. Marvin says:

    "The problem is some stupid people went crying to the DOJ…"

    Not stupid people but rather people coming from a background of open source, operating system ending in ux and programming language ending in va. For them backward compatibility and not breaking existing clients are (or at least used to be) not a big priority as evidenced by their own products. Coming from such background any attempt on the part of MS to not document something is viewed as simply malicious.

  11. Igor says:

    “Do people complain to Lotus…”

    Raymond, Lotus doesn’t make an operating system with undocumented dynamic libraries which only their own programs can use, so your comparison is not valid.

    [Neither is Windows, so your comparison is not valid either. -Raymond]
  12. DX says:

    His comparison is valid Raymond, You known it but I do not expect you to accept it.. ;-)

    Cheers Mate!

  13. Would the trolls go back to Slashdot please? The whole "undocumented dynamic libraries which only [Microsoft’s] own programs can use" thing is misinformed FUD to begin with.

  14. PatriotB says:

    Re: the 64-bit issue with this function… couldn’t it have been changed to operate on an array of INT_PTRs?  It would have no effect when compiling for x86, and would enable the function to work correctly in 64-bit.  I imagine the time to convert INT to INT_PTR would have been trivial.

    Re: DirectUI… I’ve mentioned it before on this blog.  There’s two sides to the coin: 1: Someone at Microsoft has developed some technology, and since it is property of Microsoft, any product in Microsoft should have the right to incorporate that source code.  2: Microsoft doesn’t want to commit to a publicly-available (and supported forever) implementation of the technology, so to sidestep around antitrust requirements, it doesn’t implement it as an exposed API but simply duplicates all the code in each relevant product.

    My personal view? I’d love to see DirectUI documented, because it seems like cool technology (for those who don’t want to use WPF for the same reasons that Windows itself doesn’t want to use WPF).  But I think they have the right to keep it private; as Raymond said it doesn’t use any undocumented Windows APIs so in theory any ISV could have, with the right skills and knowledge, created the same thing.

    That said, just because you *can* keep it private, doesn’t mean you should.  And that leads to…

    Re: APIs and helper functions/controls.  I’m one who loves APIs because they allow me to 1) minimize the cost of developing my own apps, and 2) be most consistent with the OS.

    It’s a balance between ISV usefulness, platform longevity, and affect on cost (for both the ISV and Microsoft).

    With XP, I was definitely seeing a trend about reducing helper functions and exposed controls (especially compared to the ComCtl32 updates brought by IE3/4).  However with Vista, I was pleasantly surprised by what I perceived as an excellent set of new Shell APIs (not quite as much with comctl32 since there isn’t much new there).  There are hundreds of new shell/property system functions/interfaces, ranging from large pieces like ExplorerBrowser and NamespaceTreeControl, to simple helper functions like SHCreateItemXXX.  There’s also some good SDK samples.  (However, the SDK documentation itself is still quite sparse.)

    That said there are still pieces missing–we waited for years for SHCreateShellFolderView/Ex (and company) to be officially documented.  It took the DOJ to bring it about, but it finally happened in 2002.  Now there’s deprecation warnings on nearly all of these "settlement" APIs, which disappoints, but doesn’t surprise, me.

    Anyways, overall it’s a balance to find which APIs are the correct ones to expose.  Generally Microsoft does a pretty good job in determining this balance.

  15. Int19 says:

    “Misinformed FUD?”

    Well just take a peak on the loaded DLLs of “MS-Windows XP
    Add/Remove Programs” applet on the Control Panel, if you do so (with
    the help of Process Explorer for example) you will see that the
    software (which starts with the help of  the well known
    “rundll.exe”) uses the “duser.dll”.

    Now if you check “duser.dll” exports you’ll see that it’s a much
    useful and quiet undocumented system oriented DLL (titled in it’s
    version info as “Windows DirectUser Engine” wow!!) that only MS
    software can use and benefit from ;-)

    Now, if Raymond can explain us this, then ok…

    [This is a lose-lose discussion for me.
    People just keep making false statements and my choices are either to
    let them stand and make people think they’re right, or correct them,
    and that just leads to a new false statement. So I’ll try to quit the
    game: Your statement is incorrect but I will not correct it. End of
    discussion. -Raymond
    ]
  16. Int19 says:

    It is not a "lose-lose discussion", if you want to explain this…

    Believe me, I would be glad to read your opinion on this matter and so help me correct my misunderstanding about this DLL (there is much hype on the net about it), as your book helped correct some of my mistakes on Windows API programming (by the way it’s a nice book :-)).

    But since you do not feel this way, “End of Discussion” from me also, I am sorry for pointing about “DUSER.DLL” in the first place, I will not trouble you again.

    Bye.. :-(

Comments are closed.