How can I reserve a range of address space and create nonzero memory on demand when the program reads or writes a page in the range?

Last time, we looked at how you can reserve a range of address space and receive notifications when the program first reads or writes a page in the range, in the case where you merely want the notification, or maybe just want to commit blank pages. But what if you want to create nonzero memory instead of just committing blank pages?

For example, you might want to simulate a memory-mapped file, except that instead of being backed by a file, it's backed by some algorithmically generated data. For example, you might be doing pointer swizzling, wherein a large database is incrementally loaded into memory as pages of it are faulted in. As each page faults in, each pointer on the page is updated (swizzled) to point to an as-yet unused page in the reserved region. When an access violation occurs in a reserved page, the code looks up which database page that reserved page corresponds to, loads the page from the database, and then updates each pointer on that page to point to an as-yet unused page. From the program's point of view, the database is being paged in on demand.

Pointer swizzling is particularly handy when accessing a very large database on a 32-bit system, because you don't have to memory-map the entire database. The memory usage is the number of pages actually faulted in, and the address space usage is the number of pages referenced by faulted-in pages.

You would handle an access violation on a reserved page by allocating a page of data at the desired location, reading the raw data from the database, and swizzling the pointers. You then mark the page read-only and restart the faulting instruction. ("Look again, and you might find a surprise!")

If you take a write protection violation, then you mark the page as dirty in your data structures, remove write protection from the page, and restart the faulting instruction. When the database file is closed, you unswizzle all the pointers in the dirty pages and write them back to the database.

As with the case discussed last time, you can choose between a structured exception handler (if you need this only for the duration of a function call) or a vectored exception handler (which remains active until explicitly removed). In the case of a swizzled database, you probably would install a vectored exception handler when the database is opened and remove it when the database is closed.

And as with the case discussed last time, you have to watch out for passing these buffers directly to kernel mode, because kernel mode will reject them as invalid. You'll have to turn them from pretend memory to real memory before using them as the source or destination buffer of a kernel mode function.

The nastier problem is multithreading. If one thread chases a swizzled pointer to a reserved page, your code will start filling the page with data. During that time, another thread can chase the same pointer, or another swizzled pointer to the same page, and start accessing the memory while the first thread is still getting the memory ready.

We'll take up this topic next time.

Comments (5)
  1. ranta says:

    Jumping ahead, I imagine one can handle the multithreaded case by calling MapViewOfFile twice to map two views of the same section object to different virtual addresses. The program would access the data via the first view; the exception handler would change the page protection at the corresponding address in the second view, populate the data there, and change the protection in the first view. I suppose the exception handler would need to enter a critical section as well. That makes me worry about ill-behaved antivirus software possibly causing another exception within the vectored exception handler.

  2. Peter Doubleday says:

    Is there a good reason to “simulate” a memory-mapped file that passes the -100 points test?

    1. Pointer swizzling is the example I gave.

    2. Anything that includes “the database file” has already passed +100 and beyond; when customers are willing to pay millions of dollars to avoid Oracle’s tens of millions of dollars for more assurance and a few performance optimizations, you dig into the nitty-gritty of user page faults to squeeze out every micro-optimization.

  3. Woah There says:

    Is there a way to do this technique in a way that handles COW smarter?

    e.g. if a process reads from the database the page is mapped in and swizzled. This page of memory can be evicted by the OS at any time and not backed up by the OS’s swap file since it can be deterministically generated. If another process maps in the same and reads from the same page, this page is shared between processes.

Comments are closed.

Skip to main content