Does mapping the same shared memory two times in a process lead to double the address space usage?

A customer designed a system which uses shared memory. Specifically, for each database file, they create a corresponding shared memory block of, say, 200MB. Multiple clients which connect to the same database file use the same shared memory block. Naturally, if two processes each access the same database file, each process will map the shared memory block into their respective address space. The question arose regarding what happens if one process connects to the same database file twice. Will the two calls to Map­View­Of­File share the same address space, or will each one allocate a separate chunk of address space?

Win32 makes no guarantees what will happen. All that you can be sure of is that the memory will be mapped into your address space, and you will get a pointer to it, and when you're done, you call Unmap­View­Of­File. Whether the two calls return the same pointer is unspecified.

In fact, Windows 95 returned the same pointer, whereas Windows NT returns a different pointer. We saw this earlier when we intentionally mapped the same shared memory block multiple times, and observed somebody actually taking a dependency on this behavior in order to effect the strangest way of detecting Windows NT. Don't take a dependency on this behavior; who knows, maybe a future version of Windows NT will consolidate multiple mappings in order to conserve address space.

If you want force this consolidation behavior, you'll have to roll it yourself, say with a lookup table of active mappings and a reference count.

Comments (19)
  1. John says:

    In regards to the Windows NT detection, wow.  How do you refrain from gouging your eyes when you see something like that?

  2. Erik says:

    Or better yet, what is the ugliest way you've ever seen an application detect Windows NT?

  3. Joshua says:

    @Eric: How about LoadLibrary("ntdll.dll"). I've seen spurious success on Win32s (community version).

  4. Tanveer Badar says:

    Could you like send a web request to your server, parse the UA string and spit back operating system to client application? :D

    @Joshua ntdll.dll did exist in Windows 98 at least.

  5. Mmxx says:

    Not to defend them, and also their purpose might have been nefarious.

    But probably the multimedia company who wrote that complex NT detection routine (and all the associated bugs) was trying to find a way to detect NT even in case someone used the application compatibility or other tricks to fake the reported version.

  6. Joshua says:

    @Tanveer Badar: No it doesn't. Nor Win95. But I've seen Win32s with an ntdll.dll with a couple of library functions in it (Rtl…..).

  7. Maurits says:

    TL;DR version: maybe.

  8. Robert says:

    You can force several mappings by specifying different protection options (dwDesiredAccess parameter in MapViewOfFile), and presumably modify these with VirtualProtect. Any changes to the current mapping behavior would therefore be inconsistent.

  9. @Joshua

    On the Windows 98 SE media, in WIN98_38.CAB there is the file ntdll.dll. It is around 20KB in size and it has a few of the functions exported from the NT version of ntdll (probably to help with portability). Of course, this DLL depends on kernel32.dll, not the other way around as is the case in NT.

    I don't have the original Win98 media to check that out, but it is definitely there in SE.

  10. Joshua says:

    @Crescens2k: Well that would explain it then. I don't have any 9x other than 95 anymore but I never checked 98SE.

  11. Pramod says:

    What's the real problem here? Is anybody actually running out of bytes in a 64-bit address space? Seems unbelievable to me.

    That said, I'd be careful about virtual address aliasing. I may have worked on some processors which might have shipped with performance bugs when using aliased cache lines.

  12. @Pramod

    Maybe not 64bit, but could be the case for 32bit.

    You must remember that these posts are queued for quite some time, and can be in response to things that happened even longer before that.

    So think about you being stuck back in 2004/2005 and your application is going to run on a 32bit system. So if you were mapping in multiple database files, would you want to let so much address space be used up for the same database?

  13. Gabe says:

    Why do so many people assume that everybody can just instantly switch to 64-bit? There are plenty of people with only 32-bit hardware, running 32-bit OSes on 64-bit hardware, using 32-bit only libraries, working with a codebase that isn't portable to 64-bit, or even on an architecture that doesn't have a 64-bit version (like ARM).

  14. Myria says:

    I have a program that forks off a second process.  The first process creates a pagefile-backed section with CreateFileMapping, then DuplicateHandles it into the child process.  Both processes then map the section into their address spaces with MapViewOfFileEx.  Can these two programs, using the same underlying section object, rely upon the mapping being shared with memory writes instantly available to the other side?

  15. @Myria

    From the CreateFileMapping documentation found at…/aa366537(v=vs.85).aspx

    "With one important exception, file views derived from any file mapping object that is backed by the same file are coherent or identical at a specific time. Coherency is guaranteed for views within a process and for views that are mapped by different processes.

    The exception is related to remote files. Although CreateFileMapping works with remote files, it does not keep them coherent. For example, if two computers both map a file as writable, and both change the same page, each computer only sees its own writes to the page. When the data gets updated on the disk, it is not merged.

    A mapped file and a file that is accessed by using the input and output (I/O) functions (ReadFile and WriteFile) are not necessarily coherent."

  16. Killer{R} says:

    Seems comments on…/247918.aspx cant be posted, so I post here a little offtopic: I saw another strange way to detect Win'NT – trying to GetFileAttributes("???.???") and checking GetLastError() to be eq ERROR_INVALID_NAME that causes code to assume running under 'NT (didnt check what retuned on '9x)

  17. Drak says:

    @Zan Lynx: Except that companies use computer from specific manufaturers who charge probably about 5 to 10 times as much (because they don't do upgrades, you need a whole new pc) and you'd have to do (many) more than one computer… For a smaller company that's a LOT of money.

  18. Zan Lynx says:

    Gabe, people who want to map over 2 GB of data (or people who want to run an application that requires it) can these days be expected to pay the $200 for a PC upgrade to a 64-bit system. Yeah. $100 for a 64-bit CPU, 4 GB RAM *and* a motherboard and $100 for a OEM copy of Win-7.

  19. @Zan Lynx says:

    Yes. And port third parties DLL & COM components to 64bit. And check&fix all pointer casts. And retest it all. That is – if you can – because those libraries may still be 32bit only, or some needed hardware piece might not have 64bit drivers yet, etc. etc. etc.

    Stop thinking only about your desktop.

Comments are closed.