You did it!

As many of you may know, we recently announced a pretty big change to the C# 2.0 language.  The full details of the change can be found at Soma's blog but i'll include the information here.

We designed the Nullable type to be the platform solution, a single type that all applications can rely on to uniformly represent the null state for value types. Languages like C# went ahead and built in further language features to make this new primitive feel even more at home. The idea was to blur the subtle distinction between this new value-type null and the familiar reference-type null. Yet, as it turns out, enough significant differences remained to cause quite a bit of confusion.

We soon realized the root of the problem sat in how we chose to define the Nullable type. Generics were now available in the new runtime and it seemed quite simple to use this feature to build up a new parameterized type that could easily encode both a value type and an extra flag to describe its null state. And by defining the Nullable type also as a value type we retained both the runtime behaviors and most of the performance of the underlying primitive. No need to special case anything in the runtime. We could handle it all as just an addition to the runtime libraries, or so we thought.

As several of you pointed out, the Nullable type worked well only in strongly-typed scenarios. Once an instance of the type was boxed (by casting to the base ‘Object’ type), it became a boxed value type, and no matter what its original ‘null’ state claimed, the boxed value-type was never null.

int ? x = null;

      object y = x;

if (y == null) { // oops, it is not null?

        ...

      }

It also became increasingly difficult to tell whether a variable used in a generic type or method was ever null.

void Foo<T>(T t) {

if (t == null) { // never true if T is a Nullable<S>?

       }

    }

Clearly this had to change. We had a solution in Visual Studio 2005 Beta2 that gave users static methods that could determine the correct null-ness for nullable types in these more or less ‘untyped’ scenarios. However, these methods were costly to call and difficult to remember to use. The feedback you gave us was that you expected it to simply work right by default.

So we went back to the drawing board. After looking at several different workarounds and options, it became clear to all that no amount of tweaking of the languages or framework code was ever going to get this type to work as expected.

The only viable solution was one that needed the runtime to change. To do that, it would require concerted effort by a lot of different teams working under an already constrained schedule. This was a big risk for us because so many components and products depend on the runtime that it has to be locked down much sooner than anything else. Even a small change can have significant ripple effects throughout the company, adding work and causing delays. Even the suggestion of a change caused quite a bit of turmoil. Needless to say, many were against the proposal for very credible reasons. It was a difficult decision to make.

We were fortunate that so many here were willing to put in the extra work it took to explore the change, prototyping it and testing it, that a lot of the uncertainty and angst was put to rest, making the decision to go ahead all that much easier.

The outcome is that the Nullable type is now a new basic runtime intrinsic. It is still declared as a generic value-type, yet the runtime treats it special. One of the foremost changes is that boxing now honors the null state. A Nullabe int now boxes to become not a boxed Nullable int but a boxed int (or a null reference as the null state may indicate.) Likewise, it is now possible to unbox any kind of boxed value-type into its Nullable type equivalent.

int x = 10;

  object y = x;

int ? z = (int?) y; // unbox into a Nullable<int>

Together, these changes allow you to mix and match Nullable types with boxed types in a variety of loosely typed API’s such as reflection. Each becomes an alternative, interchangeable representation of the other.

The C# language was then able to introduce additional behaviors that make the difference between the Nullable type and reference types even more seamless. For example, since boxing now removes the Nullable wrapper, boxing instead the enclosed type, other kinds of coercions that also implied boxing became interesting. It is now possible to coerce a Nullable type to an interface implemented by the enclosed type.

int ? x = 0;

       IComparable<int> ic = x; // implicit coercion

The reason i'm bringing this up is that i wanted to call out something specific that Soma mentions:

In the past, I have talked about how your feedback is a critical part of us building the right product. Recently, we took a big DCR (Design Change Request) into Visual Studio 2005 that was in response to your feedback. This was a hard call, because it was a big change that touched many components including the CLR. Nonetheless, we decided to take this change at this late stage in the game because a) this was the right product design and I always believe in optimizing for the long-term and b) I had confidence in the team(s) to be able to get this work done in time for Visual Studio 2005. This is a classic example of how we are listening to your feedback that results in a better product for all of us.

I cannot stress to you how true and honest a statement this is.  This issue would not have been addressed had it not been for the amazing feedback we recieved from some amazingly helpful people.  There were several that i can think of, but i definitely wanted to call out one person in specific:

Stuard Ballard took the time on several occasions to send us the message that our Nullable solution was unsatisfactory.  However, instead of just saying "it sucks" and leaving it at that.  He willingly engaged us and took quite a lot of time to write up a full and detailed explanation of why is sucked, and why he felt that it was an unnacceptable solution for him and the rest of the development community.  He even wrote up a great blog post on the subject that drilled down into many different areas where our Nullable implementation was unsatisfactory.  This page was sent out to the entire language design group where we discussed it on many occasions.  While we were aware of the limiations of our original Nullable implementation, we had previously existed in a sort of limbo state where we felt the problems were unfortunate, but acceptable.  And, when we were considering the cost of "doing it right", we felt that this might be a case where it was OK to get it slightly wrong since we could do it so cheaply.  Great community members like Stuart told us, unequivocally that it wasn't. 

Thanks Stuart! Thanks for letting us know that you woudn't let us settle for "good enough." With your help we'll have made the VS2005 release that much better for everybody. When it comes to C# 3.0 i hope that we'll be doing a lot more of this since the benefits are so fantastic to all.