Be a language designer redux…

Thanks for all the responses to the question that I posed. The reason I posed it is that I’ve been seeing a lot of language requests coming through the MSDN feedback site, which is a good thing, but some of them are pretty obviously impractical, so I wanted to try to explore a little of the “language designer” mindset.

Several people pointed out that Cyrus had blogged about this a while back. That’s true, though he doesn’t explore things from a language designer standpoint, but rather from a language user standpoint. That’s a nice place to start, but it doesn’t really explore the areas you need to explore when thinking about adding a language feature.

So, here’s my analysis of the request. Note that I’m only one member (so to be an ex-member) of the language design team, so there may be other opinions…

If you truly want to support non-null types, they need to be a runtime feature. That’s the way that you get a secure, easy-to-use system that works the way users would want it to work.

Unfortunately, at this point in time, it would be somewhat difficult for the runtime team to do it. Even if they did, it’s really not practical to modify all the existing libraries to take advantage of such a feature, as that would pretty much require that every bit of .NET code be revisited/recompiled. If you changed String.IndexOf() to only take a non-null string, think of how many places that’s used in the .NET universe.

If we had it to do again, creating the runtime and CLR with such support seems like a good thing to do, but we weren’t sharp enough to figure that out early on, so it’s not something we can realistically add at this point.

So, where does that leave us? Well, it is possible to approach this from the “parameter validation” perspective rather than the “not null type” perspective, and you can likely get the effect that you want (ie not having to write your own parameter validation code), though it’s decidedly less efficient to check for null everywhere than to just know that null is not a possible value. We will be talking about the parameter validation scenario in future design team meetings.

I think that Daniel O’Connell and Matthew W. Jackson probably came the closest to how I would approach the issue, so 250 points to each of them.

Comments (17)

  1. Daniel O'Connell says:

    Yay! Points.

    Quick question though, what constrains the system from using a generic type(NotNull<T>) that can be implicitly converted to T. It doesn’t operate entirely within the framework in that you would have to do (NotNull<T>) or (T!) casts when strings are returned, but I see no issue with passing the null value into a parameter.

    A further nicety would be return type overloading(and overload resolution) in C# andor VB or allowing the compiler to constrain valid methods to the set it wishes to use. Allowing return type overloading would allow the framework to slowly produce a duplicated interface which will be ignored by strictly CLS compliant consumers(since it is a generic) and would start to solve the issue. The C# compiler could simply *hide* those methods that are overloaded based on some metadata(with a compiler switch, of course). Its a touch contrived, but it may be a better, eventual solution. There is no issue with return nonnull values being assigned to nullable variables as nonnull is always correct. Parameters are the sticky point.

  2. Eric Wilson says:

    Are those like the points from, "Who’s Line is it Anyways?"

  3. Nicholas Allen says:

    I think there’s kind of a false dichotomy of perspectives here. For a statement S I can think of 6 ways to approach it:

    1) Hard runtime check- The runtime environment uses a verification prover or other computation to ensure S is true.

    2) Soft runtime check- The reference compiler emits a check to ensure S is true.

    3) Definite compiler check- The compiler asserts that S is true when needed but does not emit a check.

    4) Indefinite compiler check- The compiler asserts that S is definitely not false.

    5) Active documentation- An annotation is provided such that an intelligent editor can remind that S should be true and guide the user.

    6) Passive documentation- A canonical way of stating that S should be true.

    I know that language designers love options 1 and 2, but I think 3 and 6 have very good scenarios for this as well (and I’m sure the IDE people would be willing to champion 5). There seems to be some premature binning by starting out thinking about how to put this into the runtime.

  4. Daniel O'Connell says:

    Personally, I like 3 with metadata and existing runtime expectations(using a valuetype allows non-nullability without any runtime mods). Basically its a library solution that needs some language help, like Nullable Types. Though I suppose ! syntax would count as 6, would it not?

  5. Matthew W. Jackson says:

    I just got back from Fry’s and they wouldn’t take my points as currency.

    Anyway, I have another problem with using

    NonNullable<T> where T : class { … }

    as the implementation of a non-nullable type.

    Structs in C# have an implied default constructor which constructs the type with all the fields zeroed. While the compiler warns if you create a struct local and don’t explictly call a constructor, this wouldn’t stop anyone from calling the default constructor directly.

    While it is possible to create a default constructor for a struct that doesn’t set all the bits to zero, C# doesn’t allow it.

    So let’s assume that the writers of the BCL could get around this hurdle, and there was no default constructor for the struct, or perhaps the default constructor was private so it could not be called directly.

    This would prevent the following code:

    NonNullable<string> myString = new NonNullable<string>();

    Seems like a workable solution at first.

    But there’s still a problem with the ability to create a struct with all its fields zeroed. Arrays of structs by default do not call the default constructor for each element unless you call Array.Initialize(). Presumably, one could get a null-reference to an object even while using NonNullable in the following manner:

    NonNullable<string>[] stringArray = new NonNullable<string>[1];

    NonNullable<string> myString = stringArray[0];


    And suddenly SomeMethod is broken, since it’s not expecting the string that myString wraps to have a null reference.

    So what’s the solution? Do you not allow arrays of NonNullable<T>? That seems like a bad idea, because I can see situations where an array of non-nullable values could be useful Otherwise, List<NonNullable<T>> wouldn’t work.

    Calling Array.Initialize() implicitly won’t help because there’s no way to define a meaningful default constructor in this case. This is complicated when using generic types because you can’t possibly locate all code that will create an array of non-nullable values until the closed Generic type is created.

    Even then, arrays can be created using Array.CreateInstance(…). This method would have to be changed to include a special case for NonNullable<T>.

    This gets complicated for both the languague and the BCL designers.

    This is why I favor the parameter-decorating method which simply generates code at the beginning of the method to verify the parameters.

    Perhaps the original problem could be somewhat solved by modifying the runtime so that if the code generated to check a parameter using an attribute would throw exceptions in the context of the caller, not the callee.

    Then if we defined the following method:

    public void Foo([NotNull] string text) { }

    And called it with:


    The exception would be thrown on the line "Foo(null)", and the stack trace would not even indicate that the exception was thrown from inside Foo. A debugger could possibly even highlight the argument in question. This shifts the "blame" from the method which was simply validating what is passed into it to the method that passed the wrong parameters in the first place.

  6. Daniel O'Connell says:

    Code generation doesn’t solve the problem as stated. It only solves *a* problem. The original isuse wasn’t making null checks easier, but making passing null values illegal. Also, making it parameter only just seems to be a huge waste. Without local variables and definate non-null assignment, I dn’t think the feature should be added.

    As for the array issue, thats a catch I hadn’t thought of. That does make NonNullable<T> an incomplete solution. Time to put more thought into this, it seems, ;).

  7. Joe White says:

    You *could* have arrays of non-nullables. You could do:

    string[] x = new string[20];

    // … fill x[] …

    string![] y = (string![]) x;

    (Same way you can cast an object[] to a string[], with the same runtime typechecks). Or:

    string![] x = new string[] { "foo", "bar", "baz" };

    But not new string![20], which does put a crimp in List<string!>. Though that could be fixed with a syntax that lets you strip any ! off of T — something like:

    public class List<T> {

    private StripNonNullable<T>[] items;

    public this[int index]{

    get{ return (T) items[index]; }

    set{ items[index] = value; }

    // …


  8. Matthew W. Jackson says:

    The reason I’m all for pushing the attribute method is so that your code and documentation could be in sync.

    Any attributes that decorated parameters to automagically restrict their values could also be used to help generate documentation.

    If, during the development of the component, you decide that you aren’t going to allow null for a parameter, just slap on a [NonNull] and viola, your documentation now contains a blurb about how that parameter cannot be null.

    I would personally like more complicated restrictions, such as:

    public GetValue([RestrictRange(0, "Count")] int index) {


    In this case, the precompiler could generate the following code (complete with all the properties filled out on the Exception object indicating the allowed range, the name of the parameter, etc.)

    if(index < 0 || index > this.Count) {

    throw new ArgumentOutOfRangeException(…);


    I know it reeks of MACROs, but like I said it lets the method body get to the point and show what the method DOES.

  9. Matthew W. Jackson says:

    I need to proofread these:

    Correction: index >= this.Count

    I hope everyone got the idea though.

  10. The deoration of parameters w/ attributes ([NotNull] etc.) that Matthew W. Jackson talked about, is a very good example of AOP applicability, where you use attributes to assign aspects to objects & methods.

    Having AOP would mean that an idea like that could be implemented and used now as opposed to having to wait for the language design team to see it, gain momentum, implement it and ship it in n+ years.

    I personally think this is a Very Good Thing.

    Other than that, I totally agree that pushing AOP in the hands of the average application coder is a Very Bad Thing.

    Giving it to the community though, so that tool vendors can bring aspect-oriented ideas like that one to life is something different.

  11. Eric Newton says:

    isnt adding nullable types into the runtime and applying TDD to the BCL’s enough to verify the newer functionality?

    and the existing code’s verification is through the existing tests, correct?

  12. Daniel O'Connell says:

    Matthew: My biggest argument with the attribute method is I *hate* AOP thus far(if it is not *IN* code, it shouldn’t *be* code. AOP breaks that tenent on every level.) I don’t think it will actually be a good thing for anyone. Its an easy solution, but not nessecerily the best. I’d rather drop a feature than try to use something far to simple.

    While these attributes will provide A solution, including documentation, they potentially give you nasty duplicity: In language A the error occurs outside of the method, in language B it happens inside the method, in language C it could happen in either place, etc. Thats not a can of worms I would want to open. This type of functionality could be written without using attributes, by using constraint clauses or something.

    public void MyMethod(int x, string y)

    throws new ArgumentOutOfRangeException("x must be between 10 and 50",x) if ((x < 10) || (x > 50))

    throws new ArgumentNullException("y cannot be null",y) if (y == null)




    is one design I was working with to help with the exception situation(not those throws clauses aren’t all-inclusive, it was just a keyword I was using to express argument constraints and the associated exception. Its better documentatable and more flexible than attributes would be, and it actually *LOOKS* like code. And, for that matter, throws(or throw or whatever keyword) *could* be context senstive like where is simply based on its location.

    Beyond that, I don’t support attributes as language features in any circumstance. They exist for primarly library beneifit. If something can’t fit teh langauge as syntax it doesn’t belong. But that is a stance that may not be held by many(or most) of the C# developers out there.

    All of this still misses the absolute non-null ideal we should all be looking at, but it is a solution to the issue many people have raised.

  13. Matthew W. Jackson says:

    I actually do like your syntax a bit more than the attributes. The attributes make it easy for the parameter list to get really long, making it hard to see which parameters are actually needed.

    However, using a separate clause after the method definition and before the beginning of the method’s scope seems like a perfect compromise.

    Actually, I don’t *really* mind using my set of static methods to verify common scenarios. It makes it easy to solve the range problem when a number has to be between 0 and Count-1, whereas the attributes would require a kludge (a string parameter) which is far from type-safe. Generics make this easier because now I can use IComparable<T> and write one function to verify the range of any data type.

    The one gripe about static methods is that, except for the times in which they get inlined (which seems like a fairly small window), my exception stack trace starts in some seemingly arbitrary helper method, which doesn’t help me at all. I have to dig down the the second item on the trace (I know, it’s not that bad). Is there an attribute I can mark these methods with that will keep them from showing up in a stack trace? Actually, I wish I could strongly suggest they’d become inline, but the designers of .NET and C# apparently didn’t think we were responsible enough to have such a feature (well, I have seen inline functions terribly abused). Returning an exception and throwing it if it’s not null would do what I want, but that makes parameter-verifying code longer, which defeats the purpose. (Unless the language/framework were changed to allow throwing a null to be the same as not throwing anything).

    Oh, and by the way, I don’t really like AOP either. And it’s not because I don’t like other paradigms, either. I’m very fond of functional, logical, prototype, etc. languages, but something bugs me about AOP and I can’t put my finger on it. I just think that if it becomes widespread it’s going to blow up in our face.

    I think the issue we’re talking about now doesn’t need AOP as much as some sort of design-by-contract, but we don’t need that to its fullest extent, either. Most people (myself including) prefer unit testing to DBC.

    Anyway, I think that I’m going to look into writing some attributes to decorate the methods so I can automatically document the restrictions on the parameters, but mark them with a ConditionalAttribute("DOCUMENT") so that they don’t bloat my code. Then I can write a tool similar to FxCop that makes sure that when I document a parameter that I’m actually checking it with one of my helper methods…Or perhaps extend my unit testing so that it automatically tests those parameters…I don’t know, but I have a few ideas to run with. I guess I could come up with a custom schema for the XML comments to do this instead of attributes. So many options, so little free time to test them.

  14. Daniel O'Connell says:

    I don’t particularly mind seperate methods either, however declarative functionality would come in handy. The heavier question is interface based constraints, etc. How do you design an interface with restrictions, it’d have to be a set of metadata, it would be potentially unpleasent to say the least.

    One major piece of the throws design would be automatic unit tests to validate parameters. Your attributexml doc system would probably do the job as well. I would probably use xml docs myself.

  15. Matthew W. Jackson says:

    In case anyone is still reading these discussions…

    I’ve considered another pseudo-solution, that would require a few changes to the framework but shouldn’t break code-compatibility. I’ll have to think about binary-compatiblity, which it may or may not ruin. It would change a few behaviors of the framework in spots, and I’m not really happy with it for a few reasons I’ll discuss. But I thought I’d put it out there, since we’re discussing a complicated problem.

    Since the most common problem with null arguments lies in the String class, and there is a subtle (but potentially dangerous) difference between an empty string and a null string, I throw out the following (non-)solution.

    What if you could mark certain instance methods as safe to be called from a null reference? All of the string instance methods, for example, could act on a null reference in the same logical manner as it would act on an empty string.

    For example:

    string s = null;



    //this block wouldn’t execute


    The Contains method would need to be changed so that it would compare the "this" pointer to null, and if it were null, act accordingly.


    I’m not certain, but I think if you "call" an instance method (not "callvirt"), the runtime will not check for a null reference, and will call the method anyway. As long as the method doesn’t read or write any private instance fields, it will execute without incident (this is similar to how it would work on C++, I believe).

    However, for a null instance to be used for non-static methods, "callvirt" would have to change its behavior, since it checks for null references and throws exceptions early. I’m not sure how changing its behavior would work. It would probably have to check for null after resolving the vtable. I don’t like the performance implications. And how would this behave through interfaces?

    By the way, I believe that C# never seems to emit "call" instruction unless it can guarantee the reference is not null, and this is only the case when calling methods on ValueTypes and when calling methods on the base class using "base". I was trying to get C# to generate a "call" instruction to see if I could make an instance method execute on a null reference and do something like modify a static field (which wouldn’t cause a NullReferenceException), but nothing I do causes C# to emit calls in a case where this could happen.

    Anyway, that’s my solution to the problem, which doesn’t really solve anything, and…yeah.