Security got a lot of attention in Vista. UAC, LoRIE, Session 0 isolation are all prime examples. There are other areas that don’t get as much attention, but the OS implementation has changed and things that used to work in XP days, all of a sudden break in unexpected areas with unexpected errors. We found one the other day in RPC.
[Disclaimer: I am not even remotely an RPC expert and I must admit that I don’t know if I regret that. The only IDL I have modified was ATL generated and it was mostly limited to filling out the function descriptions.]
If you’re familiar with Secure CRT and the new printf_s family of functions, it will sound somewhat familiar. Those new functions have an additional parameter that allows the caller to pass in the size of the buffer. This is obviously done to prevent buffer overflows.
In RPC, you can pass a buffer as an [out] parameter. This means that when your RPC client gets data back from the RPC server, RPC runtime will use that buffer to stuff the data in. There are options how you specify the length of the [out] buffer. The recommended way is to use the [size_is] attribute (comp. the additional parameter to printf_s) and pass in the size as an additional parameter. However this is not mandatory. You can leave that parameter of and just pass in a pointer. If you leave that pointer NULL, RPC will allocate the buffer for you and fill out the pointer. Works as expected. However, if you preallocated the buffer and you omit the size_is, it becomes interesting.
RPC has no way of knowing how big the buffer is. To prevent Buffer Overflows, it will do the only thing it can and look for a closing character: ‘\0’ in this case. It checks for it with strlen (or one of its peers for Unicode, etc). If the data returned is too much for the buffer you will get this RPC Exception:
0x000006c6 means RPC_S_INVALID_BOUND.
Call stack will look something like this.
There are some interesting consequences. If you memset the output buffer to 0 you will get the exception. If you have an [in, out] buffer to pass a string and return a string, the [in] string has to be longer or equal to the [out] string, otherwise same problem.
You should of course use the size_is parameter to never be impacted. If you inherited a code base with signatures that omit the size_is parameter, you can opt for setting the pointer to NULL and let RPC do the work. With [in, out] parameters you will have to be creative.