What do HeapLock and HeapUnlock do, and when do I need to call them?


You never need to call the Heap­Lock and Heap­Unlock functions under normal operation. Assuming the heap is serialized (which is the default), all the standard heap functions like Heap­Allocate and Heap­Free will automatically serialize.

In fact, the way they serialize is by calling the¹ Heap­Lock and Heap­Unlock functions!

Nearly all heap operations complete in a single call. If your heap is serialized, this means that the heap operation takes the heap lock, does its work, and then releases the heap lock and returns. If all heap operations were like this, then there would be no need for Heap­Lock or Heap­Unlock.

Unfortunately, there is also the Heap­Walk function, which does a little bit of work, and then returns with a partial result. The design for Heap­Walk is that the application calls the function repeatedly until it either gets all the results it wants, or gets bored and gives up. But wait, what if the heap changes while the application is trying to walk through it? To prevent that from happening, the program can call Heap­Lock before starting the enumeration, and Heap­Unlock when it is done. During the time the heap is locked, other threads which attempt to call a Heap­Xxx function with that same heap will block until the heap is unlocked.

The ability to lock the heap creates a lot of potential for craziness, because the heap is a high-traffic area. As a result, it is very important that any code which calls Heap­Lock do very little while the lock is held. Take the lock, do your thing, and get out quickly.

But wait, there's more. Holding the heap lock blocks all other threads from allocating or freeing memory. This puts the heap lock very low in your lock hierarchy. Therefore, while you hold the heap lock, you cannot block on synchronization objects whose owners might try to access the heap you just locked. Consider the following:

// Code in italics is wrong.
void BadIdea()
{
 HeapLock(GetProcessHeap());
 SendMessage(...);
 HeapUnlock(GetProcessHeap());
}

Sending a message is a big deal. The thread that is the target of the message may be waiting for the heap lock, and now you've created a deadlock. You won't proceed until that thread processes the message, but that thread can't process the message until you unlock the heap.

You might accidentally do something wrong while hold the heap lock if you happen to trigger a delay-loaded DLL, in which case your call into that other DLL turns into a call to Load­Library, and now you've lost control. In practice, the only thing you should be doing while holding the heap lock is calling Heap­Walk and saving the results locally, and in a way that doesn't allocate or free memory on the heap you are walking! Wait until after you unlock the heap to start studying the results you collected or transfer the raw data into a more suitable data structure.

Bonus chatter

Note that if you call Heap­Lock or Heap­Unlock on a heap that was created without serialization (HEAP_NO_SERIALIZATION), then the results are undefined. That's because passing the HEAP_NO_SERIALIZATION flag means "Hey, Heap Manager, don't bother locking this heap. I will take responsibility for ensuring that only one thread operates on this heap at a time." If you later call Heap­Lock on a no-serialization heap, the heap manager will say, "Wha? You said that you would take care of serialization, not me!"

It's like ordering a car and saying, "Don't bother installing door locks. I will take responsibility for ensuring the safety of the car. (Say, by never letting the car leave a secured facility.)" And then a month later, calling OnStar and saying, "Hi, can you remotely lock my car for me? Thanks." Dude, you explicitly opted out of door locks.

(Amazingly, I encountered one developer who thought that calling Heap­Lock on a no-serialization heap would cause other heap operations on the heap to be blocked, even if they passed the HEAP_NO_SERIALIZATION flag to those operations. Um, no, the Heap­Lock function cannot lock a no-serialization heap because a no-serialization heap doesn't have lock in the first place, at your request.)

Nitpicker's corner

¹ s/the/the functional equivalents of/

Comments (16)
  1. henke37 says:

    Next up on crazy shenanigans found in code: manually digging up a reference to the heap lock and manually locking the heap.

  2. Maurits says:

    Should be s/the/functional equivalents of the/

  3. "Amazingly, I encountered one developer who thought that calling Heap­Lock on a no-serialization heap would cause other heap operations on the heap to be blocked, even if they passed the HEAP_NO_SERIALIZATION flag to those operations."

    Well that's not the biggest crime in the world.  They sound similar, but I wouldn't necessarily know until I looked it up in the documentation.

  4. DSN says:

    Just from the name, it is not unreasonable to assume that the HEAP_NO_SERIALIZATION flag means that there is no automatic serialization, and that any required serialization must use the lock/unlock functions.  

  5. Antonio Rodríguez says:

    @Veltas: IMHO, you shouldn't use the HEAP_NO_SERIALIZATION flag (or any other one) without reading the documentation about it… I know, I know, it usually goes like this: 1) Write code 2) Test it 3) If it doesn't work, look up documentation. But it shouldn't.

  6. voo says:

    @DSN I agree, that was actually my assumption as well before reading the documentation: HEAP_NO_SERIALIZATION means don't give me any synchronization except if explicitly demanded by the Lock function.

  7. asdbsd says:

    @DSN: On the other hand, if HeapLock on NO_SERIALIZATION heap worked, it would give someone false impression that they still somehow locked the heap (it's called HeapLock after all). Maybe it really is better to fail and make sure people just can't do anything ambiguous.

  8. Leibovich says:

    It would be nice if the library would abort the program if it request locking the heap after passing HEAP_NO_LOCK, if the overhead isn't too great. Better fail early than later.

  9. NT says:

    It would appear that your comments section doesn't load properly on android 2.3, leading me to, for the second time in the past week, see that nobody had commented and offer some brilliant insight only to then go visit the page on my desktop machine and discover that my insight was less unique than I originally thought.  I stand by its brilliance, though.

  10. cburn11 says:

    @NT I have the same e/perien e with the stock android browser. Mozilla`s Android browser seems to display the comments correctly though-

  11. Wiseass says:

    I want to nitpick the comment that led to the nitpicker's corner, and point out that his correction changes way too many lines :)

    "You never need to call functional equivalents of the HeapLock and HeapUnlock functions under normal operation."

    "But wait, functional equivalents of there's more. Holding the heap lock blocks all other threads…"

    :D

  12. Joshua says:

    I actually used HEAP_NO_SERIALIZATION once, on a private heap that was being used as an arena allocator so I didn't have to free memory in error condition (the normal path freed its memory anyway as it happened to avoid excessive memory consumption). It also had the nice side effect of making the code trivially provably leak free.

    I suppose there could be another reasonable use of HEAP_NO_SERIALIZATION besides known single-thread heap but I'm not currently aware of one.

  13. I agree mostly with all the comments about Raymond's comment "I encountered one developer who thought that…". In summary, holding beliefs about the behaviour of a call without reading the documentation, especially about a non-standard flag you decided to use, is most certainly poor practice. I don't agree with Raymond's tone implying that it is laughable, though.

    [The laughable part was following the logic through to its conclusion: "If I manually call Heap­Lock, then that means that even when I pass the HEAP_NO_SERIALIZATION flag, the call will wait for the heap lock. Which means that all heap operations block on the heap lock even if I pass the no-serialization flag. In other words, the no-serialization flag has no effect." -Raymond]
  14. Gabe says:

    I think the incorrect mental model that allowed HeapLock to work with HEAP_NO_SERIALIZATION was that asking for no serialization actually used a R/W lock. Normal operations invoke the R-lock while HeapLock invokes the W-lock. This model would allow a HeapLock operation to block while a HeapAlloc or HeapFree were going on (or another operation protected by HeapLock), while not serializing HeapAlloc or HeapFree operations.

    Of course, I can't explain why you would want those semantics or why somebody would assume them.

    [Of course, HeapLock/Unlock existed decades before R/W locks, and W locks are not recursive, so you end up deadlocking against yourself. And I still don't understand why people think "If I pass the no-serialization flag, I will get serialization (just not as much as usual)." What would be the point of partial serialization? (Concurrent access to a no-serialization heap is expressly forbidden.) -Raymond]
  15. NT says:

    Yes, that would be my expectation.  I agree that the expectation that other calls will be blocked is bogus.  You ask for no-serialization, you do the serialization yourself.  I would just have expected to be able to implement my own serialization using HeapLock and HeapUnlock.  Guess not.

  16. NT says:

    The thinking isn't "If I pass the no-serialization flag, I will get serialization (just not as much as usual)," it's "If I pass the no-serialization flag, no serialization is done *for me*, so obviously I need to manually lock and unlock the heap."  What's not obvious is that "no serialization" means "don't serialize (and also make manually calling lock/unlock fail to do anything)."

    [Okay, I can see how one might think that the 'no serialization' flag means "Do not call Heap­Lock automatically. I have to call it manually if I want to serialize heap access." But then where does the expectation that calling Heap­Lock will block other calls into the heap (even if they pass the no-serialization flag)? Wouldn't the expectation be "I have to call Heap­Lock manually if I want to serialize heap access"? -Raymond]

Comments are closed.