Hardcore pointer marshalling samples for Windows CE 6

This is one of those blogs where I hesitated to post it because it may be going too hard core into the guts of CE.  But from dealing with our customers who use our shared source and some of the really hard core questions I get, I'm hoping there are at least 5 super-geeks who can benefit from this.

Suppose you're writing a device driver or a service that has to do all sorts of really gross pointer marshalling across process.  For instance, consider MSMQ.  Apps get at MSMQ by calling API's like MQOpenQueue in msmqrt.dll.  This function is called from user space, but just takes the paramaters the user specified, calls DeviceIoControl(hHandleToMSMQ,...), which calls the MSMQ service to do the real heavy lifting.

In the old kernel with the flat address space this was easy (relatively).  In the new VM model it gets trickier.  Sue Loh has a good blog about this at https://blogs.msdn.com/ce_base/archive/2006/11/09/Memory-marshalling-in-Windows-CE.aspx.

Sue Loh has another good blog about the raw pointer marshal API for CE 6 at https://blogs.msdn.com/ce_base/archive/2006/11/22/marshalling-helper-apis.aspx

This API was designed so that both kernel level drivers and user mode drivers (e.g. udevice.exe & servicesd.exe) could use the same set of API's.  To make marshaling even easier, we have a set of templates in the %_WINCEROOT%\public\common\oak\inc\psl_marshaler.hxx.  This header is extensively commented along with samples showing how to make it work.  The upshot is that if you have some user-mode API that takes say 5 paramaters (some ptrs that need to be marshalled across process, others regular old DWORDs) then by using the PSL Marshaller templates you can save yourself a lot of the tedium as far as manually packing and unpacking these datatypes during the DeviceIoControl() call.

Besides the psl_marshaler code itself, some samples may help if you have to do this.

GPSID:  GPS Intermediate Driver.  This is the easiest sample (by far) I know of that uses psl_marshaler stuff in actual shipping code.

%_WINCEROOT%\public\COMMON\oak\drivers\GPSID\client - GPSAPI.dll src, link time library.  All it does is just call to the psl_marshall helpers
%_WINCEROOT%\public\COMMON\oak\drivers\GPSID\service - GPSID.dll, actual service that implements guts of this.

You can look on https://msdn.microsoft.com/library/en-us/mobilesdk5/html/mob5grfGPSIntermediateDriverFunctions.asp?frame=true to get a sense that there's just not that much going on with this API.  Actually that's intentional to keep implementation easier :).  The harder ones are code & API definitions we port from desktop Windows.  Here they have MIDL to do all the really heavy lifting for them, unlike CE where it's still psuedo-manual.  CE actually does have interprocess COM and we could rely on it & MIDL to do our IPC, but it adds 500KB to our ROM, isn't on Windows Mobile devices, and isn't as efficient in terms of CPU/memory as using the raw access routines & templates.

MSMQ: Microsoft Message Queuing.  Note you'll need to be a Platform Builder customer and install the shared source kit in order to get this code.
We're past the point of the mythical 5 people who actually need this much information.  I'm hoping you guys can figure out how to do the interprocess calls based on docs, blog entries, and the GPSID sample.  Now I'm basically targetting 1 person who actually has to marshall pointers inside structures inside arrays of VARIANTs, and to do it asyncronously at that.  My first recommendation is that if you have any control over your API, then try to keep it as simple as possible.  No embedded pointers or arrays, no asyncronous writing back of data.

However, if you're in this situation you're where I was at when getting MSMQ up on the new kernel.  In this case I'll point you at the MSMQ code in hopes that it may provide you some inspiration as to what you're trying to do.

%_WINCEROOT%\private\servers\msmq\sc\ce\client -> msmqrt.dll, runtime library apps link to to call into MSMQ.
%_WINCEROOT%\private\servers\msmq\sc\ce\driver\msmqdev.cxx - MSMQD.dll.  Especially MMQ_IOControl().

[Author: John Spaith]