A history of GlobalLock, part 4: A peek at the implementation


On one of our internal discussion mailing lists, someone posted the following question:

We have some code that was using DragQueryFile to extract file paths. The prototype for DragQueryFile appears as follows:

UINT DragQueryFile(
    HDROP hDrop,
    UINT iFile,
    LPTSTR lpszFile,
    UINT cch
);

In the code we have, instead of passing an HDROP as the first parameter, we were passing in a pointer to a DROPFILES structure. This code was working fine for the last few months until some protocol changes we made in packet layouts over the weekend.

I know that the bug is that we should be passing an HDROP handle instead of a pointer, but I am just curious as to why this worked so flawlessly until now. In other words, what determines the validity of a handle and how come a pointer can sometimes be used instead of a handle?

GlobalLock accepts HGLOBALs that refer to either GMEM_MOVEABLE or GMEM_FIXED memory. The rule for Win32 is that for fixed memory, the HGLOBAL is itself a pointer to the memory, whereas for moveable memory, the HGLOBAL is a handle that needs to be converted to a pointer.

GlobalAlloc works closely with GlobalLock so that GlobalLock can be fast. If the memory happens to be aligned just right and pass some other tests, GlobalLock says "Woo-hoo, this is a handle to a GMEM_FIXED block of memory, so I should just return the pointer back."

The packet layout changes probably altered the alignment, which in turn caused GlobalLock no longer to recognize (mistakenly) the invalid parameter as a GMEM_FIXED handle. It then went down other parts of the validation path and realized that the handle wasn't valid at all.

This is not, of course, granting permission to pass bogus pointers to GlobalLock; I'm just explaining why the problem kicked up all of a sudden even though it has always been there.

With that lead-in, what's the real story behind GMEM_MOVEABLE in Win32?

GMEM_MOVEABLE memory allocates a "handle". This handle can be converted to memory via GlobalLock. You can call GlobalReAlloc() on an unlocked GMEM_MOVEABLE block (or a locked GMEM_MOVEABLE block when you pass the GMEM_MOVEABLE flag to GlobalReAlloc which means "move it even if it's locked") and the memory will move, but the handle will continue to refer to it. You have to re-lock the handle to get the new address it got moved to.

GMEM_MOVEABLE is largely unnecessary; it provides additional functionality that most people have no use for. Most people don't mind when Realloc hands back a different value from the original. GMEM_MOVEABLE is primarily for the case where you hand out a memory handle, and then you decide to realloc it behind the handle's back. If you use GMEM_MOVEABLE, the handle remains valid even though the memory it refers to has moved.

This may sound like a neat feature, but in practice it's much more trouble than it's worth. If you decide to use moveable memory, you have to lock it before accessing it, then unlock it when done. All this lock/unlock overhead becomes a real pain, since you can't use pointers any more. You have to use handles and convert them to pointers right before you use them. (This also means no pointers into the middle of a moveable object.)

Consequently, moveable memory is useless in practice.

Note, however, that GMEM_MOVEABLE still lingers on in various places for compatibility reasons. For example, clipboard data must be allocated as moveable. If you break this rule, some programs will crash because they made undocumented assumptions about how the heap manager internally manages handles to moveable memory blocks instead of calling GlobalLock to convert the handle to a pointer.

A very common error is forgetting to lock global handles before using them. If you forget and instead just cast a moveable memory handle to a pointer, you will get strange results (and will likely corrupt the heap). Specifically, global handles passed via the hGlobal member of the STGMEDIUM structure, returned via the GetClipboardData function, as well as lesser-known places like the hDevMode and hDevNames members of the PRINTDLG structure are all potentially moveable. What's scary is that if you make this mistake, you might actually get away with it for a long time (if the memory you're looking at happened to be allocated as GMEM_FIXED), and then suddenly one day it crashes because all of a sudden somebody gave you memory that was allocated as GMEM_MOVEABLE.

Okay, that's enough about the legacy of the 16-bit memory manager for now. My head is starting to hurt...

Comments (11)
  1. AC says:

    Am I the only one who thinks it a big mistake to allow a coding error like this (using a pointer vs. handle) to work most of the time? That’s just asking for the bug to show up. Instead of just saying "don’t do this" in the docs, why not actually enforce it? (i.e., don’t ever just return the pointer back from GlobalAlloc).

  2. asdf says:

    Two questions, 1. Would the app verifier be able to spot that bug 2. is GMEM_MOVABLE’s semantics the same as GMEM_FIXED (and the GlobalFlags function removed from, etc.) in the Win64 api?

  3. Raymond Chen says:
    1. The problem is that the caller didn’t call any APIs; they just cast a handle to a pointer. AppVerifier has no hook into a compiler cast.

      2. Win64 is intended to be source-compatible with Win32. Changing function semantics would break source compatibility.

  4. Raymond Chen says:

    Then again, the correct code raises an error too if you omit the cast:

    HGLOBAL hglob = GlobalAlloc(GMEM_MOVEABLE, …);

    DROPFILES* pdrop = GlobalLock(hglob);

    pdrop->pFiles = …;

    GlobalUnlock(hglob);

    DragQueryFile(hglob, …);

    error C2664: ‘DragQueryFileA’ : cannot convert parameter 1 from ‘HGLOBAL’ to ‘HDROP’

  5. Raymond Chen says:

    Ben has the basic idea of the current implementation. But of course since it’s an implementation detail, it can change at any time so don’t go grovelling into undocumented data structures… Just call the GlobalLock function and be a good citizen.

  6. Eric TF Bat says:

    "C is a language that combines all the elegance and power of assembly language with all the readability and maintainability of assembly language."

    Or, for even more rude quotes about this poor old language and its cousins, see http://www.sysprog.net/quotec.html.

  7. Raymond Chen says:

    If you pass GMEM_FIXED then you get a pointer back; if you pass GMEM_MOVEABLE then you get a handle. GlobalAlloc *must* return a pointer if the caller asked for a pointer.

  8. I used to know all this nightmare. I used to know off by heart which operations yielded the thread (old: SendMessage, MessageBox(); new: COM ), because in the win16 days, any yield would move memory around. I even remember before STRICT was enabled, and everything mapped to the same UINT, so there was no guarantee that anything would work, especially as the GUI didnt do any param validation.

    But I have erased all that stuff from my brain and filled it up with a different set of obscure coding facts. Long term, these will be equally irrelevant, but it is so good not to have to worry about pointers *at all*.

    The miracle is that some pretty good stuff shipped in those days -remember the early Excel? It worked, it even had a macro language built into the cells that wasnt viewed as a virus transmission mechanism. I am not sure that we developers have given the users as much as they deserve, given how much easier our life has become, now that the cache of a CPU is greater than all the memory that Win16+himem could address.

  9. Tom says:

    If they’d have compiled with STRICT, theyø’d have got this error:

    error C2664: ‘DragQueryFileA’ : cannot convert parameter 1 from ‘struct _DROPFILES *’ to ‘struct HDROP__ *’

  10. Ben Hutchings says:

    Ooh, I think I get it. A "handle" to a fixed block is a 16-byte-aligned pointer, but a real handle to a moveable block points to a pointer that’s offset 4 bytes into an 8-byte-aligned entry. So it’s just an alignment test:

    LPVOID GlobalLock(HGLOBAL hMem)

    {

    if (((DWORD)hMem & 0xF) == 0)

    return (LPVOID)hMem;

    /* This may not be right for Win64. Also the real implementation may detect and handle overflow somehow. */

    InterlockedIncrement((DWORD *)hMem – 1);

    return *(LPVOID *)hMem;

    }

  11. Your Mom says:

    Umm, Honey… I dont even know what your talking about… but just remember… thinking hard makes you mess yourself, remember that time you, nevermind…Love, Your Mom.

Comments are closed.

Skip to main content