Creating a shared memory block that can grow in size


A little-known feature of shared memory blocks in Win32 is that it is possible to resize them, sort of.

When you create a shared memory block, you can pass the SEC_RESERVE flag to Create­File­Mapping, then the size you pass to the function is treated as a maximum rather than an exact size. (Don't forget that Create­File­Mapping is used for creating both memory-mapped files and for creating plain old shared memory. The name of the function is misleading unless you're wearing kernel-colored glasses.)

When you map this shared memory block, you are reserving address space, but no memory is committed yet. You call Virtual­Alloc to commit memory into the shared memory block.

This means that you can create a growable shared memory block by creating an initially empty block, and then committing a small amount of memory into it. When you want to grow the block, you commit more. However, you cannot shrink the shared memory block. Once the memory is committed, it cannot be decommitted.

Here's a demonstration. Note that most error checking has been elided for expository purposes. Note also that since the memory isn't actually being shared with anybody, this program working too hard; it could have just used plain old Virtual­Alloc. So pretend that the memory is being shared with somebody else.

#include <windows.h>
#include <stdio.h>

#define ONE_GIGABYTE (1024 * 1024 * 1024)
#define VIEW_SIZE (ONE_GIGABYTE / 2) // We will map half of it

void ReportMemoryPresence(void *p)
{
 MEMORY_BASIC_INFORMATION mbi;
 VirtualQuery(p, &mbi, sizeof(mbi));
 printf("Memory at %p is %s\n", p,
        (mbi.State & MEM_COMMIT) ? "committed" : "not committed");
}

void WaitForEnter()
{
 char dummy[64];
 fgets(dummy, 64, stdin);
}

int __cdecl wmain(int, wchar_t **)
{
 BYTE *pView;
 HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, NULL,
                              PAGE_READWRITE,
                              0, VIEW_SIZE,
                              NULL);
 printf("Created the file mapping\n");
 WaitForEnter();

 pView = (BYTE*)MapViewOfFile(h, FILE_MAP_WRITE, 0, 0, VIEW_SIZE);
 printf("Mapped half of it at %p\n", pView);

 ReportMemoryPresence(pView);
 ReportMemoryPresence(pView + VIEW_SIZE - 1);
 WaitForEnter();

 return 0;
}

In this version, we create a one-gigabyte shared memory block with no special flags, which means that all the memory gets committed up front. When you run this program, it reports that the memory at the start and end of the mapping is present. That's because the normal mode for shared memory is to commit it all at creation.

You can watch the effect of commit by running Task Manager, going to the Performance tab, and looking at the value under Committed. It should jump by a gigabyte when "Created the file mapping" is printed. (For some reason, the Commit size in the Details pane counts the view as commitment, even though the view consists almost entirely of reserved rather than committed pages.)

Now let's add the SEC_RESERVE flag:

 HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, NULL,
                              PAGE_READWRITE | SEC_RESERVE,
                              0, VIEW_SIZE,
                              NULL);

Now when you run the program, Task Manager's Committed memory does not increase. That's because we created an empty shared memory block with the potential to grow up to one gigabyte, but right now it is size zero. This is confirmed by the memory presence check, which reports that the memory at the start and end of the mapped view is not committed.

Okay, well, a zero-length shared memory block isn't very useful, so let's make it, say, 100 megabytes in size.

#define BLOCK_SIZE (100 * 1024 * 1024)

int __cdecl wmain(int, wchar_t **)
{
 BYTE *pView;
 HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, NULL,
                              PAGE_READWRITE | SEC_RESERVE,
                              0, VIEW_SIZE,
                              NULL);
 printf("Created the file mapping\n");
 WaitForEnter();

 pView = (BYTE*)MapViewOfFile(h, FILE_MAP_WRITE, 0, 0, VIEW_SIZE);
 printf("Mapped half of it at %p\n", pView);

 ReportMemoryPresence(pView);
 ReportMemoryPresence(pView + VIEW_SIZE - 1);
 WaitForEnter();

 VirtualAlloc(pView, BLOCK_SIZE, MEM_COMMIT, PAGE_READWRITE);
 printf("Committed some of it at %p\n", pView);

 ReportMemoryPresence(pView);
 ReportMemoryPresence(pView + BLOCK_SIZE - 1);
 ReportMemoryPresence(pView + BLOCK_SIZE);
 ReportMemoryPresence(pView + VIEW_SIZE - 1);
 WaitForEnter();

 return 0;
}

Watch the Committed memory in Task Manager go up by 0.1 gigabytes when we commit some of it. Also observe that the memory presence checks show that we have exactly 100 megabytes of memory available; the byte at 100 megabytes + 1 is not present.

Okay, so we were able to grow the shared memory block from zero to 100 megabytes. Let's grow it again up to 200 megabytes.

int __cdecl wmain(int, wchar_t **)
{
 ...

 VirtualAlloc(pView + BLOCK_SIZE, BLOCK_SIZE, MEM_COMMIT, PAGE_READWRITE);
 printf("Committed some of it at %p\n", pView + BLOCK_SIZE);

 ReportMemoryPresence(pView);
 ReportMemoryPresence(pView + 2 * BLOCK_SIZE - 1);
 ReportMemoryPresence(pView + 2 * BLOCK_SIZE);
 ReportMemoryPresence(pView + VIEW_SIZE - 1);
 WaitForEnter();

 return 0;
}

Okay, well there you go, a growable shared memory block. If you wanted to conserve address space, you could use Map­View­Of­File to map only the number of bytes you intend to commit, and each time you want to grow the memory block, you create a new larger view. I didn't bother with that because I'm lazy.

Bonus chatter: Another way to get the effect of growable and shrinkable shared memory blocks is to cheat and create multiple shared memory blocks, but map them right next to each other.

Bonus chatter 2: You can get sort of the effect of decommitting memory from the block by resetting it (MEM_RESET). The memory is still committed, but you told the memory manager that if the memory needs to be paged out, just discard it rather than writing it to disk.

Bonus chatter 3: Be aware that creating very large SEC_RESERVE sections can incur high commit charges for the page tables themselves. This is significantly improved in Windows 8.1, which defers committing the page tables until you actually use them.

Comments (13)
  1. Pierre B. says:

    Unless there is something I'm completely missing, the code actually only create a half-gigabyte file mapping. I think your call to CreateFileMapping() should pass ONE_GIGABYTE instead of VIEW_SIZE to match the explanatory text.

  2. Rick C says:

    "Note that most error checking has been elided for expository purposes."

    Everyone always says that.  Have you ever considered putting in all the error checking, to demonstrate _that_?

    [That was my old policy and it resulted in code that people pointed to and laughed at because it was so wordy and heavily indented. -Raymond]
  3. SimonRev says:

    Mostly that would demonstrate what a pain error codes are and why I personally prefer exceptions.  (Been trawling through a bunch of COM examples lately, and I am completely sick of code that looks like this

    hr = DoSomeCOMThing()

    if (!SUCCEEDED(hr))

       goto CleanupBlock7;

  4. alegr1 says:

    But what happens, if you have a reserved section mapped in two processes, and one process commits some memory to it. Does that memory become accessible in the second process without it having to commit?

  5. Jan Ringoš says:

    I am particularly curious about the overhead of File Mapping Objects when only a few pages are reserved/committed. That's because I am using this technique (67 kB big mapping object) to share some data (3 kB) and to pass filenames around in the remaining 1 kB which can grow up to 64 kB in case someone decides to be clever with their directory depth and naming.

    But maybe I care about memory usage too much.

  6. Torkell says:

    <I>For some reason, the Commit size in the Details pane counts the view as commitment, even though the view consists almost entirely of reserved rather than committed pages</I>

    My guess is that it counts as commitment in that the kernel has guaranteed that there is enough memory available for the entire 1GB. It may not have actually allocated any physical pages or pagefile space yet, but it's promised that it will always be able to do so. I think copy-on-write mappings do something similar in that it makes sure there's enough space available for the copied pages, even if they're never needed.

    Compare with Linux where depending on configuration it's possible for allocations to succeed even if there isn't enough memory available (part of the reason for this is to allow the fork/exec model to work even if there isn't enough actual memory to fork the parent process – forking uses copy-on-write semantics). This works as long as you don't try to, say, fork a large process (over half your total memory) and then have the child change all its copied pages – if you do, then the out-of-memory killer starts up and picks a process to blat. It doesn't always choose wisely (one I saw was when it killed cron – we only found out about it several months later when the log partition filled as log files weren't being rotated anymore!).

  7. rs says:

    Talking about little-known features of shared memory blocks, is there any way to replace a shared memory blocked mapped at an address with another shared memory block mapped at the same address? (What I mean is a combination of UnmapViewOfFile and MapViewOfFileEx without the risk of some other part of the process taking away the address between the calls.)

    This could be very useful for moving large portions of memory or restoring earlier versions of memory without transferring any data.

  8. Joshua says:

    @Raymond, SimonRev: Yeah I feel your pain. Makes me love blocks that can't fail.

  9. Joshua says:

    @rs: How scary of a solution you want? I've got one that would probably make Raymond recoil in horror.

  10. Nitpicker (Corner?) says:

    Yes Joshua, more horror please! :)

    (blahblahblah, avoid spam filter)

  11. Joshua says:

    Disclaimer: This should work. I haven't tried it. It's only really interesting as a use case of SuspendThread and ResumeThread in "normal" development as opposed to manipulating somebody else's process.

    Need two DLLs (henseforth known as A and B). B has no other purpose than being a helper for A and so nobody links against it. B is linked against A. Raymond calls this the Hired Lacky. We are going to be using the hired lacky to run code with the loader lock held.

    Exported API in A ReplaceMappedViewOfFile does the following:

    1. Take lock.

    2. Place call parameters in global variables.

    3. Load DLL B with LoadLibrary.

    In B's DllMain:

    4. Stop all threads but this one. This requires CreateToolhelp32Snapshot, Thread32First, Thread32Next, and NtQuerySystemInformation (to tell if a thread is suspended already), and SuspendThread. New threads can't start because we have the loader lock.

    5. UnmapViewOfFile (reading global variables stashed in #2)

    6. MapViewOfFile

    7. Call ResumeThread on all threads suspended in step 4.

    8. Return for DllMain

    9. FreeLibrary for DLL B.

    10. Release lock.

  12. Nitpicker (Corner?) says:

    Wow, that IS horrible. I'll wake up in a cold sweat at 1am tomorrow morning. :)

    I was envisaging the sledgehammer approach of reserving as much high address space as possible, and then mapping the view with MEM_TOPDOWN – assuming you can even call MapViewOfFile with that flag…

  13. Myria says:

    I wish that MapViewOfFileEx had as many options as the real underlying API, NtMapViewOfSection.  Mapping into other processes, MEM_TOP_DOWN, inheritance control and forcing addresses below 2 GB are four features that NtMapViewOfSection has that MapViewOfFileEx does not.  NtMapViewOfSection is semi-documented; see its "Zw" variant ZwMapViewOfSection in the "Windows Dev Center – Hardware" area of MSDN.

Comments are closed.

Skip to main content