Marshalling Helper APIs

Article
11/22/2006

I am told that our APIs are not part of our documentation. :-( I know for sure we documented these, but I'm told there is a documentation update coming soon, so they must only have made it into the update. My apologies on behalf of Microsoft. Keep an eye out for update notifications inside Platform Builder / Visual Studio.

In the meantime, to help you out, I am posting the comments from some of our code. This is part of %_WINCEROOT%\private\winceos\coreos\core\thunks which does not appear to be part of our shared source. :-(

// Access-checks and marshals a buffer pointer from the source process, so

// that it may be accessed by the current process. Returns the marshalled

// pointer. This function allocates resources which must be freed by a

// subsequent call to CeCloseCallerBuffer.

// Duplication prevents asynchronous modification of the buffer by the caller.

// If duplication is not required for security purposes, don't use it. Then

// CeOpenCallerBuffer can select the most efficient marshalling method for

// best performance.

// If duplication is required, allocates a new heap buffer, copies data from the

// source buffer to the heap buffer [if necessary due to "in" descriptors

// ARG_I* or ARG_IO*], and returns the new heap buffer. If duplication is not

// required, CeOpenCallerBuffer may still duplicate the buffer, or it may

// allocate virtual address space in the current process (VirtualAlloc) and

// point it at the caller process' memory (VirtualCopy) to create an alias to

// the same memory. In all cases, any required write-back to the source buffer

// will be managed by CeCloseCallerBuffer [if necessary due to "out" descriptors

// ARG_IO* or ARG_O*].

// This call uses ReadProcessMemory and WriteProcessMemory to do its work. If

// your code is running at a low enough privilege level that it does not have

// access to those APIs, this call will fail with E_ACCESSDENIED.

// Does not allocate any resources if the call fails, or if the source buffer

// was NULL. If this call fails for any reason, the pointer returned in

// *ppDestMarshalled is NULL.

// This function opens the caller buffer for synchronous access during an API

// call. You must call CeAllocAsynchronousBuffer in order to use the buffer

// returned by CeOpenCallerBuffer asynchronously. Do not close the caller

// buffer until after you have called CeFreeAsynchronousBuffer.

// This function is protected by __try/__except so as not to throw an exception

// while accessing the input pointer pSrcUnmarshalled.

// Possible return values:

// E_INVALIDARG pSrcUnmarshalled was NULL, the length was 0, or some other

// argument was invalid.

// E_ACCESSDENIED The source buffer was an invalid address, or your code does

// not have sufficient privilege to access the memory.

// E_OUTOFMEMORY The memory allocation failed.

// S_OK The allocation (and duplication, if necessary) succeeded.

// It is strongly recommended to use the SUCCEEDED() / FAILED() macros to test

// the return value of this function.

HRESULT

CeOpenCallerBuffer(

PVOID* ppDestMarshalled, // Receives a pointer that the current

// process can use to access the buffer

// synchronously.

PVOID pSrcUnmarshalled, // Pointer to the caller's data,

// to be access checked, marshalled, and

// possibly duplicated.

DWORD cbSrc, // Size of the caller's buffer, in bytes.

// If the ArgumentDescriptor is a WSTR

// or ASTR, then a size of 0 can be used.

// If the size of a string is non-zero, then

// it must include the terminating NULL.

DWORD ArgumentDescriptor, // Descriptor explaining what kind of API

// argument the buffer is, eg. ARG_I_WSTR,

// ARG_O_PTR, etc. ARG_DW is NOT a valid

// descriptor for marshalling!

BOOL ForceDuplicate // Set to TRUE to require a temporary heap

// buffer to be allocated in the current

// process. Set to FALSE to allow

// CeOpenCallerBuffer to select the most

// efficient marshalling method.

)

// Frees any resources that were allocated by CeOpenCallerBuffer.

// Performs any required write-back to the caller buffer. (Due to "out"

// descriptors ARG_IO* or ARG_O*)

// This function is protected by __try/__except so as not to throw an exception

// while accessing the input pointer pSrcUnmarshalled.

// Possible return values:

// E_INVALIDARG pSrcUnmarshalled was NULL, the length was 0, or some other

// argument was invalid.

// E_ACCESSDENIED Required write-back could not be performed. If this error

// occurs, resources are still released and the marshalled

// pointer is no longer accessible.

// S_OK The free succeeded.

// It is strongly recommended to use the SUCCEEDED() / FAILED() macros to test

// the return value of this function.

HRESULT

CeCloseCallerBuffer(

PVOID pDestMarshalled, // Pointer to the buffer that was allocated by

// CeOpenCallerBuffer.

PVOID pSrcUnmarshalled, // The source pointer that was passed to

// CeOpenCallerBuffer.

DWORD cbSrc, // The buffer size that was passed to

// CeOpenCallerBuffer.

DWORD ArgumentDescriptor // The descriptor that was passed to

// CeOpenCallerBuffer.

)

// Re-marshals a buffer that was already marshalled by CeOpenCallerBuffer, so

// that the server can use it asynchronously after the API call has returned.

// Call this function synchronously before your API call returns. You can not

// call this function asynchronously. This function allocates resources which

// must be freed by a subsequent call to CeFreeAsynchronousBuffer.

// API parameter access (KERNEL MODE ONLY):

// You can use CeAllocAsynchronousBuffer to get asynchronous access to an API

// parameter (which would have already been marshalled by the kernel).

// However if there is any chance that your code will run in user mode, then

// don't do this. Instead follow the user mode instructions below.

// API parameter access (USER MODE):

// To access an API parameter asynchronously, define the API function

// signature so that the parameter is declared as an ARG_DW value, so that

// the kernel does not automatically marshal the parameter for you. Then call

// CeOpenCallerBuffer to marshal the parameter. The asynchronous buffer will

// become inaccessible if you close the marshaled buffer by calling

// CeCloseCallerBuffer, so you should call CeFreeAsynchronousBuffer before

// calling CeCloseCallerBuffer. In other words, do not call

// CeCloseCallerBuffer until after you have called CeFreeAsynchronousBuffer.

// CeAllocAsynchronousBuffer is not required for buffers that have been

// duplicated by CeAllocDuplicateBuffer. You do not need to do anything in

// order to use those buffers asynchronously. Those buffers can be used until

// they are closed/freed. But if you choose to call CeAllocAsynchronousBuffer

// on a duplicated buffer, it will work. In that case you must not call

// CeFreeDuplicateBuffer until after you have called CeFreeAsynchronousBuffer.

// Does not allocate any resources if the call fails, or if the source buffer

// was NULL. If duplication is required but no memory is allocated, the pointer

// returned by CeFreeAsynchronousBuffer is NULL.

// This function is protected by __try/__except so as not to throw an exception

// while accessing the input pointer pSrcSyncMarshalled.

// Possible return values:

// E_INVALIDARG pSrcUnmarshalled was NULL, or the length was 0.

// E_ACCESSDENIED The source buffer was an invalid address.

// E_OUTOFMEMORY The memory allocation failed.

// S_OK The allocation (and duplication, if necessary) succeeded.

// It is strongly recommended to use the SUCCEEDED() / FAILED() macros to test

// the return value of this function.

HRESULT

CeAllocAsynchronousBuffer(

PVOID* ppDestAsyncMarshalled, // Receives a pointer that the current

// process can use to access the buffer

// asynchronously.

PVOID pSrcSyncMarshalled, // Pointer to the buffer that has already

// been marshalled for synchronous access

// by the current process.

DWORD cbSrc, // Size of the marshalled buffer, in bytes.

// If the ArgumentDescriptor is a WSTR

// or ASTR, then a size of 0 can be used.

// If the size of a string is non-zero, then

// it must include the terminating NULL.

DWORD ArgumentDescriptor // Descriptor explaining what kind of API

// argument the buffer is, eg. ARG_I_WSTR,

// ARG_O_PTR, etc. ARG_DW is NOT a valid

// descriptor for marshalling!

)

// Frees any resources that were allocated by CeAllocAsynchronousBuffer.

// Performs any required write-back to the source buffer. (Due to "out"

// descriptors ARG_IO* or ARG_O*)

// This function is protected by __try/__except so as not to throw an exception

// while accessing the input pointer pSrcSyncMarshalled.

// Possible return values:

// E_FAIL Required write-back could not be performed. If this error

// occurs, resources are still released and the marshalled

// pointer is no longer accessible.

// S_OK The allocation (and duplication, if necessary) succeeded.

// It is strongly recommended to use the SUCCEEDED() / FAILED() macros to test

// the return value of this function.

HRESULT

CeFreeAsynchronousBuffer(

PVOID pDestAsyncMarshalled,// Pointer to the buffer that was allocated by

// CeAllocAsynchronousBuffer.

PVOID pSrcSyncMarshalled, // The source pointer that was passed to

// CeAllocAsynchronousBuffer.

DWORD cbSrc, // The buffer size that was passed to

// CeAllocAsynchronousBuffer.

DWORD ArgumentDescriptor // The descriptor that was passed to

// CeAllocAsynchronousBuffer.

)

// Flushes any changed data between source and destination buffer allocated by

// CeAllocAsynchronousBuffer.

// ARG_O_PTR: Writes back data from asynchronous buffer into source buffer.

// ARG_IO_PTR: Writes back data from asynchronous buffer into source buffer.

// Does NOT read from source buffer.

// Others: Fail with ERROR_NOT_SUPPORTED

// This function is protected by __try/__except so as not to throw an exception

// while accessing the input pointer pSrcSyncMarshalled.

// Possible return values:

// E_FAIL Required read or write-back could not be performed.

// S_OK The read or write succeeded.

// It is strongly recommended to use the SUCCEEDED() / FAILED() macros to test

// the return value of this function.

HRESULT

CeFlushAsynchronousBuffer(

PVOID pDestAsyncMarshalled,// Pointer to the buffer that was allocated by

// CeAllocAsynchronousBuffer.

PVOID pSrcSyncMarshalled, // The source pointer that was passed to

// CeAllocAsynchronousBuffer.

PVOID pSrcUnmarshalled, // The source pointer that was passed to

// CeOpenCallerBuffer, or NULL if the buffer

// was an API parameter than never came from

// CeOpenCallerBuffer (kernel mode only).

DWORD cbSrc, // The buffer size that was passed to

// CeAllocAsynchronousBuffer.

DWORD ArgumentDescriptor // The descriptor that was passed to

// CeAllocAsynchronousBuffer.

)

// This function abstracts the work required to make secure-copies of API

// arguments. Don't use it for buffers other than API arguments. Don't

// expect the duplicated buffer to be accessible after the API call returns.

// Allocates a new heap buffer, copies data from the source buffer to the heap

// buffer [if necessary due to "in" descriptors ARG_I* or ARG_IO*], and returns

// the new heap buffer. This function allocates resources which must be freed

// by a subsequent call to CeFreeDuplicateBuffer. Any required write-back to

// the source buffer will be managed by CeFreeDuplicateBuffer [if necessary due

// to "out" descriptors ARG_IO* or ARG_O*].

// Duplication prevents asynchronous modification of the buffer by the caller.

// If duplication is not required for security purposes, don't use it. Just

// access the caller's buffer as passed to your API.

// Does not allocate any memory if the call fails, or if the source buffer was

// NULL. If no memory is allocated, the pointer returned in *ppDestDuplicate

// is NULL.

// Do not use CeAllocDuplicateBuffer with a buffer marshalled by

// CeOpenCallerBuffer. Instead have CeOpenCallerBuffer do the duplication.

// You do not need to call CeAllocAsynchronousBuffer in order to use the buffer

// returned by CeAllocDuplicateBuffer asynchronously. The duplicate buffer can

// be used until it is closed by CeCloseCallerBuffer. CeAllocAsynchronousBuffer

// will not work on buffers that have been duplicated by CeAllocDuplicateBuffer.

// This function is protected by __try/__except so as not to throw an exception

// while accessing the input pointer pSrcMarshalled.

// Possible return values:

// E_INVALIDARG pSrcMarshalled was NULL, or the length was 0.

// E_ACCESSDENIED The source buffer was an invalid address, possibly a pointer

// that has not been marshalled.

// E_OUTOFMEMORY The memory allocation failed.

// S_OK The allocation and copy succeeded.

// It is strongly recommended to use the SUCCEEDED() / FAILED() macros to test

// the return value of this function.

HRESULT

CeAllocDuplicateBuffer(

PVOID* ppDestDuplicate, // Receives a pointer to a newly-allocated heap

// buffer.

PVOID pSrcMarshalled, // Pointer to the caller's data, that has

// already been marshalled.

DWORD cbSrc, // Size of the caller's buffer, in bytes.

// If the ArgumentDescriptor is a WSTR

// or ASTR, then a size of 0 can be used.

// If the size of a string is non-zero, then

// it must include the terminating NULL.

DWORD ArgumentDescriptor // Descriptor explaining what kind of API

// argument the buffer is, eg. ARG_I_WSTR,

// ARG_O_PTR, etc. ARG_DW is NOT a valid

// descriptor for duplicating!

)

// Frees a duplicate buffer that was allocated by CeAllocDuplicateBuffer.

// Performs any required write-back to the source buffer. (Due to "out"

// descriptors ARG_IO* or ARG_O*)

// This function is protected by __try/__except so as not to throw an exception

// while accessing the input pointer pSrcMarshalled.

// Possible return values:

// E_FAIL Required write-back could not be performed. If this error

// occurs, resources are still released and the duplicated

// pointer is no longer accessible.

// S_OK The free (and write-back, if necessary) succeeded.

// It is strongly recommended to use the SUCCEEDED() / FAILED() macros to test

// the return value of this function.

HRESULT

CeFreeDuplicateBuffer(

PVOID pDestDuplicate, // Pointer to the buffer that was allocated by

// CeAllocDuplicateBuffer.

PVOID pSrcMarshalled, // The source pointer that was passed to

// CeAllocDuplicateBuffer.

DWORD cbSrc, // The buffer size that was passed to

// CeAllocDuplicateBuffer.

DWORD ArgumentDescriptor // The descriptor that was passed to

// CeAllocDuplicateBuffer.

)

And here are the C++ classes from %_WINCEROOT%\public\common\oak\inc\marshal.hpp:

// This class is a wrapper for CeOpenCallerBuffer / CeCloseCallerBuffer and

// CeAllocAsynchronousBuffer / CeFreeAsynchronousBuffer.

// You should only use it with embedded pointers that have NOT already been

// access-checked or marshalled by the kernel. To duplicate a buffer that has

// already been marshalled, use DuplicatedBuffer_t. To gain asynchronous

// access to a buffer that has already been marshalled, use AsynchronousBuffer_t.

class MarshalledBuffer_t {

public:

// Access-checks and marshals a buffer pointer from the source process, so

// that it may be accessed by the current process. Exposes the marshalled

// pointer via the ptr() accessor. Any allocated resources related to the

// marshalling are freed only by a subsequent call to Unmarshal(), or by the

// destructor.

// Typically, you would either use the default constructor plus Marshal()

// to marshal the buffer, or you would use the marshalling constructor to

// accomplish the same task. Use the former method if you require an

// HRESULT. Similarly, you can allow the destructor to release marshalling

// resources, or use Unmarshal(). If an HRESULT is required, use the

// Unmarshal function.

MarshalledBuffer_t();

~MarshalledBuffer_t();

// Please see the description of CeOpenCallerBuffer and

// CeAllocAsynchronousBuffer for more information about the operation of

// this function.

// If marshalling fails, ptr() will return NULL and size() will return zero.

// Otherwise the marshalled buffer will be accessible via ptr() and size().

MarshalledBuffer_t(

PVOID pSrcUnmarshalled,

DWORD cbSrc,

DWORD ArgumentDescriptor,

BOOL ForceDuplicate = TRUE,

BOOL Asynchronous = FALSE

);

// Takes a const pSrcUnmarshalled, can only be used with ARG_I_* types

MarshalledBuffer_t(

PCVOID pSrcUnmarshalled,

DWORD cbSrc,

DWORD ArgumentDescriptor,

BOOL ForceDuplicate = TRUE,

BOOL Asynchronous = FALSE

);

// Please see the description of CeOpenCallerBuffer and

// CeAllocAsynchronousBuffer for more information about the operation of

// this function.

// Once a MarshalledBuffer is marshalled (using the marshalling constructor

// or the Marshal() method, it cannot be re-used by calling Marshal(), until

// after Unmarshal() is called. An attempt to do so will return

// ERROR_ALREADY_EXISTS.

// If Marshal() fails, ptr() will return NULL and size() will return zero.

// Otherwise the marshalled buffer will be accessible via ptr() and size().

HRESULT

Marshal(

PVOID pSrcUnmarshalled,

DWORD cbSrc,

DWORD ArgumentDescriptor,

BOOL ForceDuplicate = TRUE,

BOOL Asynchronous = FALSE

);

// Takes a const pSrcUnmarshalled, can only be used with ARG_I_* types

HRESULT

Marshal(

PCVOID pSrcUnmarshalled,

DWORD cbSrc,

DWORD ArgumentDescriptor,

BOOL ForceDuplicate = TRUE,

BOOL Asynchronous = FALSE

);

// Please see the description of CeFlushAsynchronousBuffer for more

// information about the operation of this function.

// If the buffer has already been unmarshalled, or if it is not an

// asynchronous buffer, Flush will fail with ERROR_INVALID_PARAMETER.

HRESULT Flush();

// Please see the description of CeCloseCallerBuffer for more information

// about the operation of this function.

// If the buffer has already been unmarshalled, Unmarshal will fail with

// ERROR_ALREADY_EXISTS.

HRESULT Unmarshal();

// Returns a pointer to the marshalled buffer, or NULL if the buffer

// has not been marshalled or has already been unmarshalled.

LPVOID ptr() const;

// Returns the size of the marshalled buffer, or zero if the buffer

// has not been marshalled or has already been unmarshalled.

DWORD size() const;

};

class DuplicatedBuffer_t;

// This class is a wrapper for CeAllocDuplicateBuffer / CeFreeDuplicateBuffer.

// It should only be called with API arguments that have already been

// access-checked and automatically marshalled (if necessary) by the kernel.

// All other duplication can be done by MarshalledBuffer_t.

// You can either call the constructor to do the duplication, or use

// the default constructor and then duplicate using the Allocate() method.

// If an HRESULT is required, use Allocate(). Similarly, you can allow the

// destructor to release the duplicate memory, or use the Free() method. If

// an HRESULT is required, use Free().

// If Allocate() fails, ptr() will return NULL and size() will return zero.

// Otherwise the duplicated buffer will be accessible via ptr() and size().

// Once a DuplicatedBuffer is allocated (using the constructor or the

// Allocate() method), it cannot be re-used by calling Allocate(), until

// after Free() is called. An attempt to do so will return

// ERROR_ALREADY_EXISTS.

// If the buffer is not currently allocated, Free() will fail with

// ERROR_ALREADY_EXISTS.

// Please see the description of CeAllocDuplicateBuffer and

// CeFreeDuplicateBuffer for more information about the operation of

// the Allocate() and Free() methods.

// Public methods are:

// DuplicatedBuffer_t();

// DuplicatedBuffer_t(PVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// DuplicatedBuffer_t(PCVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// ~DuplicatedBuffer_t();

// HRESULT Free();

// HRESULT Allocate(PVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// HRESULT Allocate(PCVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// LPVOID ptr() const;

// DWORD size() const;

class AsynchronousBuffer_t;

// This class is a wrapper for CeAllocAsynchronousBuffer /

// CeFreeAsynchronousBuffer. It is meant to be used with pointers

// already access-checked or marshalled by the kernel that require

// asynchronous buffer access. It should ONLY be used in kernel mode!

// See the description of CeAllocAsynchronousBuffer for information on how to

// access a buffer asynchronously in user mode.

// You can either call the constructor to allocate the async buffer, or use

// the default constructor and then use the Allocate() method.

// If an HRESULT is required, use Allocate(). Similarly, you can allow the

// destructor to release the async buffer, or use the Free() method. If

// an HRESULT is required, use Free().

// If Allocate() fails, ptr() will return NULL and size() will return zero.

// Otherwise the async buffer will be accessible via ptr() and size().

// Once an AsynchronousBuffer is allocated (using the constructor or the

// Allocate() method), it cannot be re-used by calling Allocate(), until

// after Free() is called. An attempt to do so will return

// ERROR_ALREADY_EXISTS.

// If the buffer is not currently allocated, Free() will fail with

// ERROR_ALREADY_EXISTS.

// Please see the description of CeAllocAsynchronousBuffer and

// CeFreeAsynchronousBuffer for more information about the operation of

// the Allocate() and Free() methods.

// Public methods are:

// AsynchronousBuffer_t();

// AsynchronousBuffer_t(PVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// AsynchronousBuffer_t(PCVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// ~AsynchronousBuffer_t();

// HRESULT Free();

// HRESULT Allocate(PVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// HRESULT Allocate(PCVOID pSrcMarshalled, DWORD cbSrc, DWORD ArgumentDescriptor);

// HRESULT Flush();

// LPVOID ptr() const;

// DWORD size() const;

Also see %_WINCEROOT%\public\common\oak\inc\pkfuncs.h for the ARG_* values to pass as an ArgumentDescriptor:

ARG_I_PTR // input only pointer, size in the next argument

ARG_I_WSTR // input only, unicode string

ARG_I_ASTR // input only, ascii string

ARG_I_PDW // input only, ptr to DWORD

ARG_O_PTR // output only pointer, size in the next argument

ARG_O_PDW // output only, pointer to DWORD

ARG_O_PI64 // output only, pointer to 64 bit value

ARG_IO_PTR // I/O pointer, size in the next argument

ARG_IO_PDW // I/O pointer to DWORD

ARG_IO_PI64 // I/O pointer to 64 bit value

Marshalling Helper APIs

Additional resources