Marshaling BSTRs in COM/Interop or P/Invoke
I saw people making mistakes with BSTR marshaling in the COM/Interop space. Unfortunately msdn does not have many good examples on this subject. I will share some specific code examples which may help. The examples are only for reference which may be made more robust as needed.
What is so special about BSTR?
As https://msdn2.microsoft.com/en-us/library/ms221105.aspx states BSTRs are special strings defined by Ole Automation for which memory is allocated by the system when users call SysAllocString/SysAllocStringLen. Memory allocated needs to be freed by a corresponding call of SysFreeString. Size of the BSTR is stored at 4 bytes before the actual BSTR pointer returned by SysAllocString/SysAllocStringLen on both 32bit and 64 bit platform.
How to marshal a BSTR in COM/Interop or P/Invoke calls?
Marshal a BSTR following the recommendation given at https://msdn2.microsoft.com/en-us/library/s9ts558h.aspx i.e. by using MarshalAsAttribute.
How to pass BSTR as an In/Out parameter?
https://msdn2.microsoft.com/en-us/library/x3txb6xc.aspx targets only simple strings (LPTSTR) where the buffer is passed to the P/Invoke call using StringBuilder. BSTR's are special in the sense that the buffer for the BSTR used will be provided by system.
So lets say if the hypothetical unmanaged P/Invoke call is
//A very good usage scenario is using IErrorInfo::GetDescription to get a BSTR
// and sending it back to the managed code
__declspec(dllexport)
void bstrtest(BSTR *x)
{
*x = SysAllocString(L"Something");
}
Then corresponding managed call should be
[DllImport("emptydll.dll")]
extern static void bstrtest(ref IntPtr dummy);
static
void Main(string[] args)
{
IntPtr dummy = IntPtr.Zero;
bstrtest(ref dummy); //get the bstr
String something = Marshal.PtrToStringBSTR(dummy); //convert it to the managed string
Console.WriteLine(something); //test we have got it
// now free the unmanaged BSTR. Do not forget this or there will be leaks.
Marshal.FreeBSTR(dummy);
}
How to marshal a structure (with reference-type fields) by reference?
Consider the following code
[StructLayout(LayoutKind.Sequential)]
public struct TestStruct
{
[
MarshalAs(UnmanagedType.I4)]
public int dumyInt;
[
MarshalAs(UnmanagedType.BStr)]
public string dummyString;
}
public static void Main()
{
TestStruct dummyTestStruct = new TestStruct();
dummyTestStruct.dumyInt = 10;
dummyTestStruct.dummyString = "stringTestStruct";
//get the size of the structure
int testStructSize = Marshal.SizeOf(dummyTestStruct);
//to pass the TestStruct as a reference we need IntPtr
IntPtr testStructRef = Marshal.AllocCoTaskMem(testStructSize);
//finally marshal the structure
Marshal.StructureToPtr(dummyTestStruct, testStructRef, false);
// now use TestStuctRef for an unmanaged function call
// free the allocated Memory
Marshal.FreeCoTaskMem(testStructRef);
}
We just leaked the memory for the BSTR inside TestStruct. Remember the BSTR field inside the structure is a reference type.
When marshaling dummyTestStruct using Marshal.StructureToPtr the Runtime called SysAllocString to allocate the memory for BSTR and the reference TestStruct.dummyString points to that BSTR. In the above code Marshal.DestroyStructure should have been called which will appropriately call SysFreeString. So following is the correct code (snippet)
Marshal.StructureToPtr(dummyTestStruct, testStructRef, false);
// now use TestStuctRef for an unmanaged function call
// free the allocated Memory
Marshal
.DestroyStructure(testStructRef, dummyTestStruct.GetType());
Marshal
.FreeCoTaskMem(testStructRef);
Key take away points:
1) The Marshal class has many useful functions for manipulating BSTR's https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemRuntimeInteropServicesMarshalClassTopic.asp
2) It is very important to mark BSTR with MarshalAsAttribute so that Runtime specifically knows how the application wants this string to be marshaled. Forgetting this will result in heap corruption with similar call stack as shown below.
00 0811e7f4 7c85079b ntdll!DbgBreakPoint
01 0811e804 7c8720c6 ntdll!RtlpPageHeapStop+0x72
02 0811e880 7c873305 ntdll!RtlpDphReportCorruptedBlock+0x199
03 0811e8b0 7c8734c3 ntdll!RtlpDphNormalHeapFree+0x32
04 0811e908 7c8766b9 ntdll!RtlpDebugPageHeapFree+0x146
05 0811e970 7c860386 ntdll!RtlDebugFreeHeap+0x1ed
06 0811ea48 7c81d77d ntdll!RtlFreeHeapSlowly+0x37
07 0811eb2c 776b83a6 ntdll!RtlFreeHeap+0x11a
08 0811eb40 776b84c4 ole32!CRetailMalloc_Free+0x1c
09 0811eb50 7a0afc1a ole32!CoTaskMemFree+0x13
0a 0811eb5c 79f281ff mscorwks!DefaultMarshalOverrides<WSTRMarshalerBase>::UnmarshalCLRToNativeByrefInOut+0x2a
2) Be careful when passing BSTR as In/Out parameter.
3) Avoid memory leaks by freeing the memory for the reference type fields in a structure.