VirtualLock locks your memory into the working set, even if your threads are blocked


Today, a correction to an earlier article on Virtual­Lock.

When you lock memory with Virtual­Lock, it will remain locked even if all your threads are blocked. As noted in the Follow-up section at the end of the referenced article, the behavior of the operating system never changed. Virtually-locked pages were never unlocked in practice. What changed is that an implementation detail was elevated to contract. The intention when Virtual­Lock was originally designed was that virtually-locked pages were potentially unlockable if the application is not running. However, the memory manager folks never got around to implementing that part. At some point, they decided that they would abandon any future intention to to do and strengthened the contract accordingly.

Mind you, Virtual­Lock does not guarantee that the same physical frame will always be assigned to the memory. The memory manager may reassign the memory to another physical frame in order to defragment memory so that it can allocate physically contiguous pages, primarily for I/O purposes, but occasionally to satisfy a large-page request. All it guarantees is that the memory will always be present.

The memory manager folks tell me that locked memory remains locked even if the application is suspended. But I don’t know whether that’s an implementation detail or contractual, so I wouldn’t run around relying on it.

Comments (12)
  1. Joshua says:

    Which is a good thing when the locked pages are for AIO requests.

    [Wait, are you saying that you have to VirtualLock all asynchronous I/O buffers? -Raymond]
  2. Grzechooo says:

    "The memory manager folks tell me that locked memory remains locked even if the application is suspended. But I don't know whether that's an implementation detail or contractual, so I wouldn't run around relying on it."

    Bet you someone wrote a program which relies on just this detail.

  3. Gabe says:

    Grzechooo: If you are locking pages, it's likely that you're doing so in order to maintain a fast response time. If your memory cache gets paged out to disk while you're sitting there waiting for a request, what is the point of having the cache in the first place?

    In other words, probably everybody who uses VirtualLock expects that their locked pages don't get paged out while blocked.

  4. Joshua says:

    [Wait, are you saying that you have to VirtualLock all asynchronous I/O buffers? -Raymond]

    No. If your AIO is hard real-time (e.g. CD burner), the fact that they remain locked when your threads are all waiting is vital.

    [Oh, I see. I thought you were talking about locking memory for asynchronous I/O requests that are currently active. But rather you are talking about memory that will be used for asynchronous I/O request that will be issued in the near future. (In other words, you used the present tense when you should have used the future tense.) -Raymond]
  5. Joshua says:

    (ref: linked article)

    [I wonder how unix debuggers work if they can't read the memory of the process being debugged. -Raymond]

    In the intervening years, there is now a policy added to *nix systems that only allows a process to be attached by its parents (parent, grandparent, etc). For not very obvious reasons, this policy has to be turned off when running Windows programs under wine.

  6. Evan says:

    @Joshua: "For not very obvious reasons, this policy has to be turned off when running Windows programs under wine."

    Or plenty of other common debugging scenarios. In my current work for example, I virtually always attach to existing processes from a non-parent rather than start the program under a debugger. (There are too many layers on top of the process I eventually want to debug.)

  7. Myria says:

    Is VirtualLock safe to use for cryptographic purposes – that is, to avoid sensitive memory being written to disk?

    In addition to that, it would be nice to have a way to tell Windows, "Please decommit this memory block if this machine hibernates". You could just handle the exception if somehow you were frozen at an inopportune time.

    [There does not appear to be any guarantee that the memory won't be written to disk while locked. As you noted, the machine may be hibernated, or it may be running in a VM that gets snapshotted. -Raymond]
  8. voo says:

    @Joshua: I may be misunderstanding you, but I used gdb before to debug running processes without any problems under Linux. So do common distributions just disable that policy by default or how does that work?

  9. Dave says:

    It seems to be contractual now, the VirtualLock manpage says that "pages are guaranteed not to be written to the pagefile while they are locked".

  10. Cowardly Anon Moose says:

    @Dave: They may still be written to the hibernation file or a VM snapshot.

  11. Cheong says:

    Proofread: Second paragraph last line: "…would abandon any future intention to [to] do…"

  12. Matt says:

    @Myria. No. If you care about memory being written to disk you either need it to be in non-paged kernel mode memory, or you need to remember that Bitlocker stops people who aren't logged in from reading data in your pagefile, and hence rendering the question as to whether it was written to disk completely moot.

Comments are closed.