Keyboard layouts aren’t like Beetlejuice – they won’t appear just because you say their name


A customer reported a bug in Windows Vista Home Editions:

We are handling a Ctrl+V keyboard event and want to interpret it in the context of a US-English keyboard.

// This keyState represents no keys pressed except for Ctrl
BYTE keyState[256] = {0};
keyState[VK_CONTROL] = 0x80;

// This is the handle for the US-English keyboard
HKL hkl = (HKL) 0x04090409;

// output variables
wchar_t outChar[2];
WORD outWord;

ToUnicodeEx('V', 47, keyState, outChar, 2, 0, hkl);
ToAsciiEx('V', 47, keyState, &outWord, 0, hkl);
VkKeyScanEx('V', hkl);

On Windows XP and versions of Windows Vista other than Home editions, the three calls all succeed, whereas on Windows Vista Home Editions, the calls fail. On the other hand, if instead of using the US-English keyboard, we use the current keyboard layout:

HKL hkl = GetKeyboardLayout(GetCurrentThreadId());

then Windows Vista Home Editions behave the same as Windows XP and non-Home editions of Vista.

This suggests that the Home Editions of Vista supports keyboard queries only for the currently active keyboard layout, which renders useless the last parameter to those three functions.

Notice how the customer's sample code just synthesizes a keyboard layout handle from thin air. While it is true that the format keyboard layout handles is documented, that doesn't mean that you can just make one up and start using it.

It's like saying, "I know that Contoso uses the email address format Firstname.Lastname@contoso.com, but I just tried to send email to Bob.Smith@contoso.com, and it bounced."

Does Bob work at Contoso?

"No. Does that matter?"

The customer's code blindly assumes that the US-English keyboard layout is loaded rather than calling Load­Keyboard­Layout to actually load it. As a result, if the keyboard layout is not loaded, the call will fail because you passed an invalid keyboard layout handle.

The customer liaison asked, "Is this documented somewhere that the HKL has to be created from only from the functions and cannot be assigned a value?"

Um, yeah, it's right there in the documentation of the hkl parameter to the To­Unicode­Ex function. (Emphasis mine.)

dwhkl [in, optional]

Type: HKL

The input locale identifier used to translate the specified code. This parameter can be any input locale identifier previously returned by the Load­Keyboard­Layout function.

Identical text appears in the documentation of the hkl parameter to the To­Ascii­Ex and Vk­Key­Scan­Ex functions as well.

The difference observed on Windows Vista Home Editions, then, is that on those systems, in the configurations the customer happens to be using, US-English is not a preloaded keyboard layout.

Comments (16)
  1. parkrrrr says:

    "But I called LoadKeyboardLayout last week and it gave me 0x04090409, so the parameter is a locale identifier that was previously returned by LoadKeyboardLayout."

  2. Adam Rosenfield says:

    I generally assume that any handle-like data type with an H prefix in its name is not something you can synthesize out of thin air, since there is some important corresponding data hidden away somewhere else (such as in user32/kernel32.dll or inside the kernel's memory space).  Sure, you can take a command line argument and convert it into a HANDLE (as in blogs.msdn.com/…/56043.aspx), but that doesn't necessarily mean you'll have a valid handle — but it's useless unless you actually inherited that handle from your parent process.

  3. Zan Lynx says:

    I had visions of Contoso HR synthesizing a new Bob.Smith on demand.

  4. Joshua says:

    @Adam Rosenfield: HKEY_LOCAL_MACHINE

    filler text to keep spam filter from catching this

  5. Adam Rosenfield says:

    @Joshue: Yes, that would be an exception.  It's well-documented where HKEY_LOCAL_MACHINE comes from and that it's safe to use, unlike (HKL)0x04090409.  Note that I said "data type" (as in HANDLE, HKEY, HGLOBAL, etc.), and HKEY_LOCAL_MACHINE and friends are objects rather than types.

  6. parkrrrr says:

    @Adam Rosenfield: (HPROCESS)(-1)

    (Remember, kids, we're trained professionals. Don't try this at home. Use GetCurrentProcess.)

  7. Ron says:

    "I had visions of Contoso HR synthesizing a new Bob.Smith on demand."

    Ha! Comment of the day! :D

  8. mpz says:

    I don't know if this is related, but one thing that drives me nuts is programs that add new keyboard layouts without consulting me. Steam and Valve games are extremely guilty of this. I usually only have the Finnish and Japanese keyboard layouts installed, but after playing say Half-Life 2, I find that the English keyboard has been added to the leftshift+leftalt keyboard switch procession. Under XP it'd also show up in Control Panel and was easy to remove, but now with Windows 7 it doesn't even show up there so I have to first add the English-US keyboard and then remove it to get rid of it.

    This infuriates me. Can this be prevented somehow?

  9. Joshua says:

    While we're on the topic of static H values, I know why programs keep being written that think InvalidateRect(NULL, …) refreshes the desktop.

    HWND_DESKTOP is defined as 0.

    Yes I know it's only for CreateWindow & friends, but it's not immediately obvious to someone who sees the definitions but not the documents, so the do InvalidateRect(HWND_DESKTOP, …) and expect everything to be repainted.

  10. Craig says:

    <devil's advocate>Ah, but the documentation only says the hkl parameter *can* come from a previous call to Load­Keyboard­Layout not that it *must*.

  11. Dominic says:

    @Craig:  It actually says the hkl parameter *can be any input locale identifier previously returned by the Load­Keyboard­Layout function*, this infers that it can not be anything not previously returned by the Load­Keyboard­Layout function.

  12. @mpz Everybody uses US-English keyboard layout. :))

  13. Jonathan says:

    I like how the customer assumes it must the Vista Home Edition, rather than dependent on installed keyboard layouts. When working on Fax software, I had a tester that opened bugs stating that they "repro only on German Windows machines", while it was the modem he happened to have hooked to those machines. I learned to translate German => Modem_X in my head.

    -Jonathan

  14. JM says:

    @Dominic: it can be any input locale identifier previously returned by the Load­Keyboard­Layout function *if you expect the function to actually do useful work for you, or anything else if you don't*. Honestly, how can we blame people for not using APIs correctly if the MSDN leaves out crucial information like this? Are these things even copyedited? I'd make it something like:

    The seventh parameter to the ToUnicodEx function, which we will name 'dwhkl' for convenience but which has no actual name when the function is called [*], must be of the type 'HKL', which will be defined if you include the file 'windows.h'.[**] To construct a value of the HKL type that will cause the ToUnicodeEx() function to behave in a manner consistent with the specification, you must call the LoadKeyboardLayout() function. Note that this function can return NULL on error. You must check for this, because NULL is not a valid value for the 'dwhkl' parameter to the ToUnicodeEx() function. If you pass a value for 'dwhkl' that is not a return value of a call to the LoadKeyboardLayout() function[***], the ToUnicodeEx() function may or may not fail, and this specification provides no guarantees about the actual effects of the function in this case."

    [*] Consult the documentation of your compiler and/or the C language specification on the topic of formal parameters for more information.

    [**] This specification makes no promises about the nature or size of the HKL type. Please use the HKL type only by including 'windows.h'.

    [***] "Call" in this context means a previous function call made by the same process.

    Of course, this is only a rough draft, many essentials are still missing (calling conventions, what a compiler is, how computers work). But it's miles better than what we've got.

    [You totally forgot to mention the critical details that no intervening UnloadKeyboardLayout call has been made in the same process with the same handle. Not even calls to UnloadKeyboardLayout with an uninitialized or improperly-initialized variable which happens to contain the same numeric value as the one returned by LoadKeyboardLayout. -Raymond]
  15. Ryan says:

    @mpz Nice to know I'm not the only person annoyed by that. I'd love to know how to get Source engine games to quit mucking around with my keyboard layout.

  16. ender says:

    I really wish the keyboard layouts didn't just appear out of thin air – in both Windows 7 and Windows 8 I randomly get switched to US English layout, even though I don't have it installed (and on Win7 I can't switch to it with Alt+Ctrl, though I can switch back to Slovenian after this happens). I only have Slovenian layout listed on Control Panel, but this doesn't prevent Windows from forcing US English on me (and apparently I'm not the only one). This didn't happen in Windows Vista and older – if the layout was removed from Control Panel there, Windows didn't switch me to it.

Comments are closed.

Skip to main content