Nullable types in C# (part 2)

I was reading the C# 2.0 language spec and trying to wrap my brain around the new support for System.Nullable<A> in the C# language. There are just a couple of parts that I keep running into and not quite getting. The first is the following set of statements:

1.
A nullable type can represent all values of its underlying type plus an additional null value. 2.
The underlying type of a nullable type may itself be a nullable type. Thus, types such as int?? and int??? are permitted, although they can represent no more values than int?.

I just don't know how to reconcile these statements. The specification for Nullable is as follows:

 
public struct Nullable<A> where A : struct {
     bool HasValue { get; }
     A Value { get; }

     public Nullable(A value);
}

The first statement says that a nullable type can represent all values of its underlying type plus an additional null value. That plus is essential. It means that given any value type you can always get one extra value to indicate something out of the normal set of values the type can be set to. For example, if your type 'A' is 'int' then you have 1 extra value that you can use outside of the regular -2^31 <-> 2^31 -1 regular values. This expressiveness is necessary because there are many cases where you just don't have a value that you can spare for 'special' meaning.

So that's all well and good. However, i then tried to merge that with the next sentence: "The underlying type of a nullable type may itself be a nullable type. Thus, types such as int?? and int??? are permitted, although they can represent no more values than int?." This doesn't seem to make much sense ot me. Nullable<A> is a value type, it has a certain range of allowable values. The first sentence implied that Nullable<Nullable<A>> would allow us to represent all values of the underlying type (Nullable<A>) plus an additional null value. However, the second claim says that Nullable<Nullable<A>> is no more expressive than Nullable<A>... I'm not sure how to reconcile these ideas especially in the light of another sentence which claims: "C#’s nullable types solve this long standing problem by providing complete and integrated support for nullable forms of all value types."

Nullable is a value type. However, it doesn't seem to be completely supported and integrated since it does not seem possible to encode an extra null value into the type through the use of a Nullable-Nullable.

Is this really a problem? Possibly, possibly not. Will this actuall affect anybody, or am I getting myself into a twist over nothing? I was thinking of places that one would use Nullable's and I discovered a scenario where this could be a problem. Consider the following code:

 
delegate bool Predicate<A>(A a);

IList<A> {
    ...
    A? Find(Predicate<A> p);
    ...
}

Find will search the list and return the first item that matches the delegate, or a nullable-A with no value if no element is found.

For example, you could use that in the following manner:

 
IList&ltint> ages = GetAges();
...
int? adult = ages.Find(delegate(int i) { return i >= 21; });

Which looks for an element in the list greater or equal to 21. That's kinda of useful, and I can see all sorts of ways that that makes many APIs much cleaner. In fact, the spec says it the best with "Another approach is to maintain boolean null indicators in separate fields or variables, but this doesn’t work well for parameters and return values". So, in essence, nullable types allow you to incode that extra bit of boolean state along with the actual value consolidating them into a nice object. (In fact, it's very similar to the IOptional type that Jay, kevin and I used when writing the lazy loader). However, consider the following more complicated case:

 
IList<int?> ages = GetAgesFromDatabase();
....
int?? unsetValue = ages.Find(delegate(int? i) { return ! i.HasValue; });

Here we're asking the question: "Does the list contain an item that has no value." I.e. does the database column of ages have a row whose value is null for that column. Now lets think about the answer to that question in the following two cases:

1.
The database contains no unset values for the age field. The Find operation will return an int?? that has no value. 2.
The database contains an unset value for the age field. The Find operation will want to return that, but as int?? is not more expressive than int? (and the int? will have no value, since that is what we searched for) it will return an int?? that has no value either

What we seem to have ended up with is an inability to form a very simple sentence. I.e. "is there a nullable that has no value." or "give me the nullable that has no value". This seems like something that one would like to express (especially when dealing with DBs), and we have explicitly removed that ability from you. It's also not clear what this means when we're dealing with different languages. Will all languages have this semantic for Nullable<A>? Or will it just be C#? What expressive power do we gain by adding this restriction? Why not have a simplified system that is totally consistant over all types, or at least over all value types? Thoughts on this would be appreciated. Do any of you have experience with other languages/libraries that have attempted to incorporate this and how they did it/if they were successful at it.