Some Notes about Mixed Types
Perhaps the most important reason to use C++ for managed code development is
that C++ fully understands managed code and existing C++ code. This positions
C++ as the ultimate language for doing managed-native interop programming. It is
worth sitting back and remembering just how important that capability is. This
allows you as the programmer to take advantage of any library that exists
whether it is a native library or a .NET library. At the same time, every work
item can be accomplished by using C++ alone. Although the CLR allows language
interop between assemblies and to some degree within an assembly, it's still
harder than using only one language.
C++ definitely makes interop easier than any other language on the .NET
platform, but it still requires effort... and intelligence. The most fundamental
thing to understand about .NET interop are differences between memory allocators.
These are the two models that must be understood: the CLR garbage collector and
traditional native heap memory managers.
Traditional native heap memory managers
Pick up any book on writing a memory manager, and you'll quickly learn how
tricky this can be. Nevertheless, when it comes down to the final steps of
squeezing out every last bit of performance, it inevitably comes down to writing
a custom memory manager. What is most common about traditional memory managers
is that the data stored in memory is never moved by the system. That is, from
the point the memory is allocated (via new, malloc, or your favorite memory
allocation function), the object stored in that memory can only be moved by the
program itself. The address range of that memory allocation is given over to the
program's control until the program releases that address range (via delete,
free, or your favorite memory deallocation function).
Even most garbage collectors follow this same behavior. That is from the
point of allocation, a memory range is under the control of the program until
the memory range is no longer referenced. At this point, a garbage collector can
free the memory. The contents of the data range are never stored elsewhere
unless the program itself moves the data.
Performance of memory allocators are measured in several ways, including the
time it takes to allocate memory, time it takes to free memory, wasted memory
used for padding between allocations, and long term working set. In many cases,
I've seen server applications run out of memory – not because there was a
memory leak, but because the heap's memory was fragmented. Memory fragmentation
means that while enough free memory exists for an allocation, not enough free
space is available in a contiguous block of memory addresses.
The CLR garbage collector
The garbage collector that comes with .NET is known as a generational,
compacting garbage collector. It's really quite sophisticated for the task of
memory management. Every time a garbage collection happens, any free space
between objects is compacted out by moving the remaining live objects to the
beginning of the heap. This means that free memory is always available in a
contiguous chunk in higher address range of the heap, which makes memory
allocation super fast.
There are a number of other characteristics that make the .NET garbage
collector world class, but they're not relevant to the eventual topic of this
discussion. What is relevant is that objects on the garbage collected heap are
moved by the system. To maintain consistency in the system, any memory address
that referenced the object is also updated by the system. Unfortunately, the
system (by which I mean the CLR) only knows about memory references that
originated from managed code (managed code is not necessarily equivalent to MSIL). This means that any reference to an
object on the garbage collected heap originating from unmanaged code will not be
updated, and thus the program could break after the garbage collector runs.
The best way to allow unmanaged code to refer to objects on the garbage
collected heap is to prevent the garbage collector from moving the object. The
feature that tells the garbage collector to leave an object alone is called
pinning. .NET actually provides several ways to pin an object, but there are
risks associated with all of them. I'll discuss this more later.
Choosing which memory allocator to use
Choosing whether to use the .NET garbage collector for an object verses using a
traditional memory allocator comes down to choosing whether to represent data
with a ref class or a native class. All ref classes take memory from the garbage
collected heap whenever they are instantiated with the MSIL newobj instruction.
All native classes take memory from the native heap, the execution stack, or
global memory.
In .NET, there is a category of types called value classes that are optimized
for efficient copying. As such, an instance of a value type can occupy memory
anywhere on the system with some restrictions. A value type can have a handle as
a member (see my discussion of the
handle
design in C++). Handles have to be visible to the CLR, so a value type that
contains a handle cannot be allocated on the native heap.
Now, we're at an interesting point in the discussion... with all these
restrictions for what memory allocator a type can be instantiated with, how do
we effectively interoperate between the two memory allocators. A typical
approach to coding is to make a type a member of another type. Let's try both
directions:
struct NativePoint {
int x, y;
};
ref struct ManagedPoint {
int x, y;
};
class C {
ManagedPoint mp;
};
ref class R {
NativePoint np;
};
If you try to compile this program, the Visual C++ 2005 compiler gives the
following errors:
t.cpp(5) : error C3265: cannot declare a managed 'mp' in an unmanaged 'C'
may not declare a global or static variable, or a member of a native type that
refers to objects in the gc heap t.cpp(5) : error C3076: 'C::mp' : you cannot embed an instance of a reference
type, 'ManagedPoint', in a native type t.cpp(9) : error C4368: cannot define 'np' as a member of managed 'R': mixed
types are not supported
As you can see, the compiler doesn't really like the code above. It doesn't
like putting the ManagedPoint in the native class C
that it even gives two errors. The third error brings up this notion of a
mixed type.
What is a mixed type?
In the language specification, I wrote the following definition for mixed types:
"A mixed type is either a native class or ref class that requires object
members, either by declaration or by inheritance, to be allocated on both the
garbage collected heap and the native heap." When writing a standard, I tend to
be terse since every word can end up being interpreted broadly. Here, I'll try
to explain the issue informally.
As I said earlier, native classes are always allocated on native heap or on
the execution stack, and likewise ref classes are always allocated on the
garbage collected heap. When the native class C tried to embed a ref class
ManagedPoint, the compiler doesn't have any place to put the object since parts
of it have to be on both the native heap and the garbage collected heap. The
same is true the other way with class R. Both native class C and ref class
R are
mixed types.
In the old syntax for writing managed code in C++, the compiler allowed a
particular kind of native class to be a member of a ref class. In C++, a class
that has no constructor, destructor, or copy semantics is called a POD type,
which stands for plain old data. It is equivalent to a C-structure. These
types can be safely copied around with memcpy which makes them more suitable to being
members of ref classes. Unfortunately, these types are used by unmanaged code
frequently, meaning that if the POD is allocated on the garbage collected heap,
the garbage collector cannot move that region of memory while unmanaged code is
accessing the object. This is accomplished with pinning. While there are several
ways to pin an object, the most accessible mechanism is a pinning pointer.
A pinning pointer works by keeping whatever it points to from moving during a
garbage collection. For a pinning pointer to be effective, it must be an active
local variable in a function somewhere on the call stack. As soon as a function
returns, a pinning pointer no longer keeps an object for moving. Given this
behavior, the C++ language design team observed this kind of code frequently.
struct Point {
int x, y;
};
ref struct R {
Point p;
};
void F(Point* ptr);
Point* G(R^ r) {
pin_ptr<Point> pinp = &r.p;
return pinp;
}
void K() {
R^ r = gcnew R;
F(G(r));
}
The function G has a terrible mistake – something we call
a "GC hole". Using the behavior from Visual C++ 2002 and Visual C++ 2003, Point is a POD so it can be embedded in R. Thus, the memory for the Point data is on the garbage collected heap. G pins the
object where the memory for the Point stored, and uses the pinning
pointer to get a native pointer to the Point. The coding error is
that G returns the native pointer and allows the pinning pointer to
become inactive. In effect, G returns a pointer to memory on the
garbage collected heap that is not pinned.
This coding mistake was so common that we were compelled to fix it. We tried
relentlessly to correct pin_ptr so that this kind of code could not
exist. Unfortunately, that requires data flow analysis which is an incomputable
problem without further constraints to the language. In the end, there was very
little we could to change pin_ptr. It became clear that allowing
PODs to be allocated on the garbage collected heap was an idea that could not
survive in the new language design.
How mixed types work in the new language design
While all the code examples thus far have compiler errors with Visual C++ 2005,
the C++ language design team wants the code to compile. In the beginning of 2003
we spent a few months writing a lengthy design paper that proposed "mixed types"
as the solution to the problems mentioned above. The proposal suggested that
whenever a mixed type occurred, the compiler would split all the "native parts"
and the "managed parts" into two pieces. The "native parts" would be allocated
on the native heap and the "managed parts" would be allocated on the garbage
collected heap. The two parts would then be connected to each other. All of this
splitting would be mostly invisible to the programmer, leaving the compiler to
do all the hard work.
We were so intrigued by the possibilities here that we went ahead and
designed ways to make it look like any native class could be allocated on the
garbage collected heap, or even get ways to have pointers to a ref class. All of
this came to be known as the C++ unified type system. The document describing
how all this works is a bit over fifty pages, and although my blog postings are
generally fairly lengthy, fifty pages is a bit too much even for my tastes.
Provided there is interest, I may spend time writing about the unified type
system over a series of articles.
Why native native classes, including PODs, should not be on the garbage
collected heap
In past releases, PODs could be on the garbage collected heap. When taking their
address, you get an interior pointer, which you would then have to pin in order to
get an unmanaged pointer. Since mixed types say that native classes are never
allocated on the garbage collected heap, we were able to assume that native
types never needed to be pinned. This assumption actually goes to the heart of
the type system. Consider this common implementation for the swap routine:
template<typename T>
void swap(T% t1, T% t2) {
T temp = t1;
t1 = t2;
t2 = temp;
}
We wanted to make it possible to replace a native reference (&) with a
tracking reference (%) without any trouble. After all, they are both pointers
underneath the covers and there isn't the same risk of leaking an unpinned
reference to unmanaged code. Simply substituting % for & works until a
conversion from an N% to an N& is necessary (in the language design, we
typically use the capital letter N to represent native classes). Consider this
simple code:
struct C {
int X;
C(int value) : X(value) {}
C(C& c) : X(c.X) {}
C& operator=(C& c) {
this->X = c.X;
return *this;
}
};
void K() {
C c1(1);
C c2(2);
swap(c1, c2);
}
Like every other native class in existing code, the copy constructor and copy
assignment operator use native references for the argument. The call with the
swap template could not succeed unless C either provided copy
functions that took tracking references or we allowed an N% to
convert to an N&. We chose to allow the conversion which was based
on the assumption that native classes would never be on the garbage collected
heap.
For the most part, all existing STL algorithms can be made to work for
managed code by simply replacing & with %. If managed
classes exposed iterators, the STL algorithms can work.
Why does Visual C++ 2005 not support mixed types?
Basically, if we had implemented everything written in the unified type system document, we would be
calling this release Visual C++ 2010. Really! We designed some pretty advanced
and complex stuff. The compiler team figured it would be more useful to release
a product sooner that provided the core language and leave out parts that could
be worked around with libraries. Just getting the fundamentals done with high
quality for Visual
C++ 2005 was a daunting task.
Going forward, we will be prioritizing work based on customer feedback and
scenarios that are blocked. Just listing out high priority scenarios for
upcoming releases could take a while, so I'll once again avoid exploring that
topic here. Regarding feedback, we are already getting
feedback that allowing PODs as members of ref classes is important. We also
received the same feedback from teams inside Microsoft who were trying to
program with the new C++ features.
Several times, we tried to narrow the set of features needed to implement
some part of mixed types. Ultimately, we ended up getting a very narrow subset of the mixed type
proposal that would work. Unfortunately, changes to the compiler in the past
half-year needed to be low risk (meaning that changes were certain to not cause
a slip to our November 7th delivery date). So far, we have been unable to mitigate the
risks involved in implementing even the simplest approach to mixed types.
How to workaround absence of mixed types
Given that Visual C++ 2005 gives errors for mixed types, the next best approach is a library. First, I'll address the case where a library solution has
existed for a while. As discussed earlier, the compiler returns errors C3265 and
C3076 when a program tries to embed a ref class inside a native class. The
gcroot template has been around for a while, which allows a native class to
effectively hold onto a handle to ref class. It is used as follows:
#include <vcclr.h>
ref struct ManagedPoint {
int x, y;
};
class C {
gcroot<ManagedPoint^> mp;
public:
C() : mp(gcnew ManagedPoint) {}
~C() { delete mp; }
};
The gcroot template doesn't make embedded semantics the default –
the ref class object still has to be created with gcnew. It's good
enough to be productive though, and there are already further derivations of
gcroot included in the libraries with Visual C++, such as auto_gcroot.
So, putting a ref class inside of a native class is solved with gcroot. What
about putting a native class in a ref class? Today, there is no library included
with Visual C++ that solves this. So... let's make one here.
The gcroot template is a wrapper for the
System::Runtime::InteropServices::GCHandle class. That's how data on the native
heap can get a reference to an object on the garbage collected heap. Going the
other way around is much easier – a pointer will do. Here is the first
rewriting of some of earlier code:
// First attempt at embedding a native class
// inside a ref class
struct NativePoint {
int x, y;
};
ref class R {
NativePoint* np;
public:
R() : np(new NativePoint) {}
~R() { delete np; }
};
The code above will certainly work in many cases, but it subject to memory
leaks when the class R is cleaned up by the garbage collector. This happens when
an instance of R is not cleaned up with a destructor, but is instead lets the
garbage collector call the finalizer. To fix that, we'll add a finalizer to the
code before:
// Second attempt at embedding a native class
// inside a ref class
struct NativePoint {
int x, y;
};
ref class R {
NativePoint* np;
public:
R() : np(new NativePoint) {}
~R() { this->!R(); }
!R() { delete np; }
};
Notice that I'm avoiding code duplication by calling R's
finalizer from R's destructor. This pattern certainly gets us
closer to working around the absence of mixed types. Nevertheless, it's clear
that as R holds onto more and more resources, maintaining the
destructor and finalizer, especially to be exception safe, will become unwieldy.
We also need to consider the very real possibility that a class might be
finalized more than once, or finalized when the constructor didn't complete
execution. To solve these problems, we're going to put the resource management
code in a separate template class:
// Third attempt at embedding a native class
// inside a ref class
template<typename T>
ref class Embedded {
T* t;
!Embedded() {
if (t != nullptr) {
delete t;
t = nullptr;
}
}
~Embedded() {
this->!Embedded();
}
public:
Embedded() : t(new T) {}
static T* operator&(Embedded% e) { return e.t; }
static T* operator->(Embedded% e) { return e.t; }
};
struct NativePoint {
int x, y;
};
ref class R {
Embedded<NativePoint> np;
};
With the Embedded template, we've finally got a library solution
for embedding native classes in a ref class. Of course, the illusion isn't
complete given that member access to the embedded native class must use the
arrow (->) operator instead of the dot (.) operator. I
think we can live with that difference until Visual C++ does introduce some
support for mixed types.
You'll note that this template is different from auto
pointer like templates in that Embedded completely owns the management of the native class –
it never gives up that ownership. The native class is created when the Embedded
class is created, and the native class is cleaned up when the Embedded class is
cleaned up.
Kenny
Kerr wrote up some notes about this same subject in
which he provided an AutoPtr template where the ownership of the resource is
more in control of the programmer using the AutoPtr template. In many ways the
AutoPtr
template is more flexible for non-POD classes as it allows for use of a non-default constructor.
AutoPtr really needs to use a finalizer though (otherwise it's
likely to leak resources). J
The last area we have so far left uninvestigated is the ability to embed
native arrays in a ref class. Previous versions of Visual C++ allowed this to
happen if the element type of the native array was a POD type, fundamental type, or a simple
value class (in other words a class that has no handles). A native array is
accessed via pointers and even is synonymous with pointers in much of the C++
type system. For all the reasons listed earlier, native arrays can no longer be
allocated on the garbage collected heap. This brings up an interesting problem –
holding onto a pointer to a native array from a ref class adds an extra level of
indirection. In some cases that can impact performance. In fact, some
languages introduced the idea of a fixed array just to solve this problem. So,
how can C++ get the same behavior given the constraints above?
C++ does let you put fundamental types and simple value classes as members of
a value class, and the CLR provides a mechanism for explicitly stating the size
of a value class. Putting that together, we have a solution –
the inline_array template written by Mark Hall and improved upon by Shaun
Miller:
template<typename T, int size>
[System::Runtime::CompilerServices::UnsafeValueType]
[System::Runtime::InteropServices::StructLayout
(
System::Runtime::InteropServices::LayoutKind::Explicit,
Size=(sizeof(T)*size)
)
]
public value struct inline_array {
private:
[System::Runtime::InteropServices::FieldOffset(0)]
T dummy_item;
public:
T% operator[](int index) {
return *((&dummy_item)+index);
}
static operator interior_ptr<T>(inline_array<T,size>% ia) {
return &ia.dummy_item;
}
};
ref class R {
inline_array<int, 10> arr;
};
This of course allows you to really embed an array of fundamental types.
Interestingly, the generated MSIL and metadata using the inline_array
template is almost exactly the same thing generated by fixed arrays in C#. It's
great to see workable library solutions show up before having to extend the
language.
To wrap up...
I know this subject requires a lot of knowledge to grok, but it does come up a
surprising amount. Most programmers moving code from previous releases of Visual
C++ to the new syntax introduced in Visual C++ 2005 are going to see these
issues the most. The fastest way to get a POD in a ref class is to use the
Embedded template.
We will be listening to customers to determine where to go next. Providing at
least some mixed type support seems necessary, but if you have anything to
say... please speak up! Sending feedback and suggestions through the
MSDN Product Feedback Center
is a great way of getting a permanent record of the request (and we really do
spend time with feedback sent there). I'm always interested in feedback too, so
don't hesitate to send notes my way.