A history of GlobalLock, part 3: Transitioning to Win32


Now that you know how the 16-bit memory manager handled the global heap, it's time to see how this got transitioned to the new 32-bit world.

The GlobalAlloc function continued to emulate all its previous moveability rules, but the return value of GlobalAlloc was no longer a selector since Win32 used the processor in "flat mode".

This means that the old trick of caching a selector and reallocating the memory out from under it no longer worked.

Moveability semantics were preserved. Memory blocks still had a lock count, even though it didn't really accomplish anything since Win32 never compacted memory. (Recall that the purpose of the lock count was to prevent memory from moving during a compaction.)

Moveable memory and locking could have been eliminated completely, if it weren't for the GlobalFlags function. This function returns several strange bits of information—now entirely irrelevant—the most troubling of which is the lock count. Consequently, the charade of locking must be maintained just in case there's some application that actually snoops at the lock count, or a program that expected the GlobalReAlloc function to fail on a locked block.

Aside from that, moveable memory gets you nothing aside from overhead.

The LocalAlloc function also carries the moveability overhead, but since local memory was never passed between DLLs in Win16, the local heap functions don't carry as much 16-bit compatibility overhead as the global heap functions. LocalAlloc is preferred over GlobalAlloc in Win32 for that reason. (Of course, many functions require a specific type of memory allocation, in which case you don't have any choice. The clipboard, for example, requires moveable global handles, and COM requires use of the task allocator.)

Next time, an insight into how locking is implemented (even though it doesn't do anything).

Comments (14)
  1. Tom says:

    QUOTE

    Consequently, the charade of locking must be maintained just in case there’s some application that actually snoops at the lock count, or a program that expected the GlobalReAlloc function to fail on a locked block.

    /QUOTE

    Does that mean that the Win32 team tried to remove locks and found that it broke some applications, or did they decide to emulate them from the start.

    How much of this stuff is determined in the design phase, and how much gets found in testing? Just curious really.

  2. Ivo says:

    Why doesn’t Win32 compact memory? The application would be able to allocate and free big GMEM_MOVEABLE blocks and not worry about fragmentation of the address space.

  3. Cooney says:

    I would imagine that, with rare exception, fragmentation isn’t an issue anymore. Each process has 4G, after all.

  4. Mike Dunn says:

    If moveable memory is a thing of the past, what about the clipboard still requires the GMEM_MOVEABLE flag?

  5. Raymond Chen says:

    Because there are apps that crash if the memory isn’t moveable.

  6. The clipboard doesn’t use the global heap anymore – nowadays, it’s based on the much more mondane shared memory. I guess that, since shared memory requires a nearly identical allocate/lock sequence, there would have been no value in devising an alternate API

  7. Andrei says:

    Cooney,

    each Win32 user process has 4Gb only on AMD64 boxes when they are run in WOW64. Normally, they have 2Gb or 3Gb of addressable virtual memory depending on whether /3Gb switch was used or not.

    Aside from that, fragmentation is an issue, you can have problems with fragmentation of the Win32 heap and fragmentation of virtual memory, depending on what memory manager you’re using.

  8. Nate says:

    Dare I ask what about non movable memory crashes certain programs? I cannot fathom what sort of practice would result in that, unless someone did something absurd like checking for the flags and deliberately crashing if they were not there.

    Then again, I bet Raymond has seen it all by now.

  9. Raymond Chen says:

    Some programs make undocumented assumptions about how moveable memory happens to be implemented. Here’s an MSDN article that describes the internals of the heap manager:

    http://msdn.microsoft.com/library/en-us/dngenlib/html/msdn_heapmm.asp

    Instead of using the documented GlobalLock function, those bad apps reach in and dereference the global handle table directly.

    You can see what goes wrong if the memory is not actually moveable.

  10. Raymond Chen says:

    Ivo: You’d have to do more than that. You would have to make sure *every* allocation (especially the small ones) is allocated moveable so that a compaction will move it. Do you really want to go back to the world of locking all memory before using it (and remembering to unlock it when done)?

    For one thing, it means no more "new" operator (since "new" allocates non-moveable memory). I doubt many programs would be willing to give up "new".

  11. Andrei: isn’t the 4GB addressable area going to cause backwards compatibility problems for Win32 processes? The reason they’re not given 3GB by default in 32-bit is because programmers were always assuming a 2GB stack i.e. the highest bit being zero.

  12. Tom says:

    Darren

    IIRC, only applications which are marked "large address aware" are give 4GB on AMD64.

    http://www.amd.com/us-en/assets/content_type/DownloadableAssets/Expand_Memory_of_32-bit_App_-_Microsoft_4GT-_6204.pdf

  13. Chep says:

    Cooney: let me tell you that CAD/CAM applications are certainly starting to feel some pressure from 2GB/4GB heap fragmentation issues.

  14. foxyshadis says:

    Specialized apps like that should build their own memory manager. There’s no reason to mess with one that does the right thing in most circumstances unless there’s a major bug or it’s being redesigned from scratch. MS might work with them, but generally won’t rewrite its OS for one or two small segments of the population.

Comments are closed.

Skip to main content