Shared Heap Pitfalls


Posted by: Sue Loh


I just saw someone on our newsgroups make a recommendation that to get past the Windows CE 32MB per-process VM limitation, you can use shared heaps to make your allocations.  I would like to explain some of the pitfalls of choosing that route.


First off, by “shared heap” we are talking about HeapCreate with HEAP_SHARED_READONLY.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wcecoreos5/html/wce50lrfHeapCreate.asp


It’s something new as of CE 5.0.  The HeapCreate call will reserve a chunk of memory from the shared memory area, and you can alloc small pieces at a time using HeapAlloc.  So, it sounds great right?  No more 32MB limit.  Why don’t people usually talk about this approach to escape the 32MB limitation on WinCE?


Shared heaps have some VERY BIG caveats.  For one, they were really intended to be used in a client/server environment, so that a server can give out memory that clients can read, but not write.  They are set up so that only trusted applications can write to them.  Untrusted applications can only read.  The documentation says that the process which created the heap is the only one which can write, but that’s not true — all kernel mode threads can write the heap and all other threads can only read the heap.  [I’ll have to get someone to fix the docs.]  On devices like PocketPC which run in all-kernel-mode, all applications are trusted.  However on SmartPhone and other devices which use a trust model that is not the case.  So, if you’re expecting to use shared heaps as if they were regular heaps, your application(s) must be trusted.


For another thing, since the memory is in the shared memory area, EVERYBODY can read it.  So on devices where security is important — don’t store anything secret there!


Finally, because of that intended client/server model, where create / alloc / write operations are only possible in a server, using it as if it was a normal heap inside an application is somewhat unsupported.  Our documentation doesn’t have a big red “no!” sign in it, but I’ll see if I can change that.  It may not really keep working in the future.  It is possible that applications which use shared heaps as a way to share writeable memory between processes will be broken in the future.  It is called “HEAP_SHARED_READONLY” after all.  If your intent is to share writeable memory between processes, then I MUST discourage you from using shared heaps.  You would be better off writing yourself a simple heap manager and using it on top of a named memory-mapped file instead.  That’s the primary way to share memory between processes in Windows CE.

Comments (9)

  1. Anonymous says:

    A doubt: Each process has a fixed 32MB slot. Why can’t we make the slot size dynamic and report the start and end address in the process control block? It might result in external fragmentation but its a small price to pay for the benefit we get. Secondly, what benfit is slot 0 giving us by keeping it as a "present workspace" slot? When some process variables are updated, they will have to be reported back to the actual process sort. If this process somehow fails( assume 1 in million chance), the actual process slot has "dirty" data. So why it is not wise to point the actual slot address and do the changes?

  2. Anonymous says:

    Sue,

    Great article! I hadn’t thought of using shared heaps to get around the terrible 32MB/proc limit, but it is good to know some of the pitfalls.

    Do you have any more suggestions for how to get around the 32MB wall? I’m currently working on a major project that is hitting this limitation in a big way. You mentioned writing a heap manager and using it on top of a named memory-mapped file… Do you have any advice on how to do that, or do you perhaps know of a "reference implementation"?

    Thanks.

  3. ce_base says:

    Hi Dast,

    I was recently reminded of the CeHeapCreate API, which would probably be a help in this case.

    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wcecoreos5/html/wce50lrfCeHeapCreate.asp

    Its allocator function could do:

    – To reserve memory, CreateFileMapping on a memory-mapped file (with no file underneath) and map a view of the whole mapping.

    – The commit could be a no-op or just touch the physical pages underneath to get them paged in.

    The free function would do nothing on decommit, and close the view/mapping on release.

    The hard part is sharing this between processes. Synchronization would probably not be trivial. In that case you’re probably best off setting up a single process to do the heap allocations & frees, that all the other processes have to call into. The "heap server" process would be using the named memory-mapped file as a heap, where all the other processes just open the memory-mapped file by name and then get pointers into it from the "heap server." All processes should open a mapping and a view so that they can access the memory.

    If I get a chance to play around with this, I’d post some sample code. But I wouldn’t count on that happening anytime soon.

    Good luck,

    Sue

  4. ce_base says:

    Hi Gursharan,

    That’s a complicated question. The answer is that the OS is rooted solidly in the 32-process 32-MB design. The address space belonging to a process is defined by what slot it’s running in, and the memory controller only allows a process access to its own slot unless it calls SetKMode or SetProcPermissions. The process identity is determined from its slot sometimes, and if a process has two slots its identity may become confused. Basically, there are many things that make it very difficult to give a process more than one slot to work in.

    We know that this VM model is getting to be quite a squeeze for people, and hope to do something about it in the future. But there are no easy answers in any of the existing OS versions.

  5. ce_base says:

    Oh, to the other part of Gursharan’s question, about why we use slot 0. That comes from the "execute in place" (XIP) design that is used on a lot of devices, where we store code and data in places that are defined at the time the image is built (desktop-side). The "makeimg" process chooses the locations that all DLLs in the MODULES section of ROM will run from (in slot 1), and the locations where their global data will go (in slot 0). EXE code can run from a predefined location in slot 0 without colliding with code from other EXEs.

    Sue

  6. Anonymous says:

    Superb answers, as expected:-)

  7. Anonymous says:

    I second that, Gursharan. Thanks, Sue!

  8. Anonymous says:

    What if we tweak the allocator function to force the memory reservation to get >2Mb and then commit a smaller chunk? The reserve would normally match the commit but in that specific case there will be a mismatch. It seems to work pretty well so far but I wander how the heap manager deals with that condition. Is there more fragmentation on the heap? Is there a degradation of the performance after running for a while?

  9. ce_base says:

    If you want to keep it as a private heap for your process, that should be fine. In fact it’s a little better about conserving RAM since you can decommit when the pages aren’t being used.

    However if you want to share the memory between processes you really should use memory-mapped files.