How an idea becomes a C# language feature?


In a recent email, a customer asked for a delegation feature. He wanted to do something like (purely theoretical syntax):


class MyClass: ITest
{
    MyHelperObject o: ITest;
}


so that he could delegate the implementation of the interface to a helper object, and have the compiler automatically generate the interface implementation methods.


My response was that we had talked about this several times in the past, but hadn’t been able to come up with a syntax that worked well.


This is a not uncommon state for us – on several occasions in the Whidbey (excuse me – “Visual Studio 2005”) design process, we tried to come up with a syntax that allowed the user to declare a trivial property (backing store, trival getter and setter) with one bit of syntax, but weren’t successfull.


I thought I’d talk about that process to give you some insight on how we approach things. I can’t be exhaustive with all of our considerations, mostly because we don’t really operate that way (there isn’t a set checklist). If you have a question on a specific feature, I am happy to consider talking about other features and decisions – see the “suggest a topic” hard link on my main page. Just note that my success rate on actually covering such topics is not exemplary. Also note that I make no attempt to answer questions in the order asked.


The process is fairly organic, in that the steps I list don’t happen sequentially, and they are often iterative. Despite the length of this post, I’ve simplified what goes on significantly.


Sniff Test


We start with what I’m going to call the “Sniff Test“, which is basically the highest level triage to determine if we think such a feature is something we would ever consider for C#.  Some features fail because they don’t fit the language philosophy, some fail because there’s no way we could ever express them in syntax.


Scenario Exploration


Our next level of attack is looking at what user problem we’re trying to solve. This typically involves us writing down (or up, on the whiteboard) the code that users are currently writing. Sometimes this is limited to the most common scenario, sometimes we’ll cover several scenarios – it all depends on what we need to know. This often happens iteratively with the next step.


Syntax Iteration


If we’ve gotten to this step, we’re usually fairly convinced that there is a user need. We don’t yet know if there is a language solution to that need. In this step, we’ll start talking about what we could do in the language to make the developer’s life easier.


This step typically works with Anders writing on the whiteboard while the rest of the design group gives suggestions and feedback. When we hit on a promising approach, we’ll explore it for a bit to see where it leads us, contrast it to other approaches we could take.


In this step, we’re asking questions such as:



  • Does this new syntax cover the important scenarios?
  • Does it really make the user’s life easier? (new syntax is not a panacea – each new feature adds complexity)
  • Is this syntax unambiguous? Is it possible for the parser to correctly deal with this new feature in the presence of all existing features? (this is a very limiting area)
  • Would this be a breaking change for existing users? (Adding new keywords are examples of breaking changes, but it’s also common for new language ambiguities and/or differences in behavior to show up because of new features)
  • Does this feature fit the philosophy of the language?
  • How will this impact tools? Can IntelliSense deal with this?
  • Is this a language feature, or would it be better done through an API, or through the IDE?
  • How have other languages approached this? Has their approach been successful?
  • Does this require new features in the runtime?
  • Does this require work in by other languages?
  • What Microsoft teams would use this? Are they on board to use it?

In a single session, we will sometimes decide to do a feature and sometimes decide to not do a feature, but more typically we’ll decide that we need to discuss it more. We go away and think about it, gather feedback from the people who read our language notes and those closest to the scenario, and then revisit it in a future meeting. A lot of the iteration is around exploring all the corner cases (some of which don’t come up until implementation) and making sure the feature is well defined, still does what we want it to do, and still is in the best interests of the developer.


It’s not uncommon for us to change our minds on the details of a feature, or even to decide upon reflection that we need to either reduce or eliminate a feature.


Structured Feedback


If we’ve gotten to this point, we’re fairly sure that we want to have the feature, and typically have an implementation (though not always). We use feedback with the C# Customer Council (a small group of customers who we work closely with), and the C# MVPs (we’ve done a little with the MVPs in the past, we’ll be doing more in the future) to see what their perspective is on the feature.


This is also the point where we work with the ECMA standards body to talk about our proposal, though it’s often difficult to align the schedules of the standards body with product milestones.


There may also be internal Microsoft presentations at this point, depending on the feature.


Many times the feedback is positive, but sometimes it causes us to refocus the feature, and there have been language features that we cut due to feedback at this point.


Manufacturing and Shipping


If we’ve gotten this far, the feature is in the language, and often implemented. The remainder of the work is to polish the implementation, finish the testing (sometimes testing is concurent with implementation, sometimes it lags a bit – it depends on the type of feature and our resource constraints), do the spec, and discuss how we’ll be presenting it to customers.

Comments (29)

  1. Rick Byers says:

    Thanks for the awesome post Eric. I’d be interested in hearing more detail about the sorts of things that cause features to be rejected. Is it common to reject a feature that you think would be valuable only because of syntactic compatibility limitations (parser ambiguity, breaking change, etc)?

    What are you thoughts on how language evolution should work in general (outside the confines of C#)? Do you think it would be possible to have languages that could more readily accept the type of extensions you’ve wanted to make to C# but couldn’t?

    For example, do you think there would be value in a language that added a layer of abstraction between the syntax presented to a user, and the persisted form? Eg. if a language was stored on disk as an XML representation of the parse tree, then you could evolve the language (add keywords, etc.) and rely on the IDE tools to intelligent present the code to the user.

  2. dak says:

    Is there any particular reason why this needs to be approached at the language level, at all? Unless I’m misunderstanding the desired feature, it looks like a prime candidate for the IDE to just write the code for you and wrap it in a #region block to keep it out of the way.

  3. Jelle Druyts says:

    Very interesting post, but I’m actually quite puzzled by your last sentence (although it might just be a misunderstanding): do you only create a spec for the feature *after* pretty much everything has been wrapped up already? I’d expect that to be one of the first things to put to paper and bring along in the iteration… Or isn’t the spec the feature definition in your book?

  4. Eric says:

    Jelle,

    The language spec is one of the final outputs. Our language design notes are the intermediate working document.

    It’s not worthwhile to spend the time to go to full spec format before we’ve shaken out most of the issues. Sometimes that happens before we’re fully done, sometimes after we’re design complete.

  5. Kazuhiko Kikuchi says:

    I wating this Feature. so good.

    I have written only many functions which return S_OK, in order to write a COM component.

    a code becomes short and becomes really easy.

  6. David says:

    Seems like the "proxies" notation in the Nosica language : http://www.nosica.net/documentation/nosicaspec.html#id2437992

  7. Andrew Tweddle says:

    Delphi has this feature. But its syntax is certainly a bit unusual, so I guess Borland had the same problem. Extract from the help file:

    type

    TMyClass = class(TObject, IMyInterface)

    FMyInterface: IMyInterface;

    property MyInterface: IMyInterface

    read FMyInterface implements IMyInterface;

    end;

  8. Andrew Tweddle says:

    I’m sure using an attribute would have been discussed as a solution:

    class MyClass: ITest

    {

    [InterfaceImplementor(typeof(ITest))]

    MyHelperObject o;

    }

    Any insights into why this is a bad idea?

  9. hakank.blogg says:

    How an idea becomes a C# language feature?. (Via Lambda the Ultimate.)…

  10. Daniel O'Connell says:

    Andrew: While I’m obviously not a dev on the team, I still think that using attributes to define language features is probably among the worst choices one could make.

    Attributes apply metadata, not direct functionality. An attribute should only be used when the applicable metadata needs to be accessed by the runtime or programs executing in that runtime, not for the compiler. Functionality that the runtime itself provides is one thing, but it just doesn’t fit the language model.

    When writing syntax, your goal is to express your concepts as clearly as possible. IMHO, using attributes like this is not particularly clear as it *looks* like metadata attachment while it is actually a language feature(infact its using a language feature to implement another language feature, which is probably worse).

    I am more curious about the problems with the derivation type syntax, field : interface and about what the other possibilties were. However, I certainly wouldn’t expect to find out, those questions seem to go unanswered most of the time.

    Actually, as a whole, I am curious about the ideas the team shot down, both feature and syntax wise. As it stands I know more than enough about what exists (and what will in 2, with a few minor points), but I don’t know what was considered and the reasons for abandoning syntaxes. Alas, that again is not something I expect we’ll ever see.

  11. Chris says:

    Please stop adding features. C# was such a nice language in .NET 1.0. Now in Whidbey it looks like another C++. Why do language designers always end up making languages more complicated than they need to be?

  12. Pascal says:

    Instead of adding features one by one you could add a more general mechanism that allows users to add these features on their own. See http://www.cs.ubc.ca/labs/spl/projects/elide/ for an example.

  13. Andrew Tweddle says:

    Daniel: I agree with you completely.

    However, doesn’t .Net already confuse meta-data with compiler directives in quite a few places e.g. the [Flags] attribute.

    So the Attibute concept has already lost its virginity. People can no longer think of it as just meta-data, however desirable that might once have been.

    Why not at least take advantage of the syntactic elegance of attributes (at the cost of conceptual elegance, since that has already been sacrificed elsewhere in the framework)?

  14. andrew: How is the [Flags] attribute a compiler directive?

  15. Andrew Tweddle says:

    Cyrus:

    My apologies – I think you’re right.

    .Net documentation for FlagsAttribute says:

    Bit fields can be combined using a bitwise OR operation, whereas enumerated constants cannot.

    I had assumed this meant that the compiler alters its behaviour based on the [Flags] attribute – i.e. its like a compiler directive.

    But a quick test program shows this not to be the case.

    The ToString() method operates differently, of course, but that’s entirely consistent with [Flags] being used as meta-data only.

  16. Daniel O'Connell says:

    Oh, I forgot

    Pascal: Personally, I’d like my C# to actually look like someone elses. Elide looks like a fancy way of reintroducing the #define problems we had with C++.

  17. Daniel O'Connell says:

    Andrew: There are a few situations where the compiler uses attributes to change compilation(Conditional and Obsolete come to mind), however they are both rather rare things to use and things that all languages should have access to. Those two are probably the most glaring uses, everything else is pretty much runtime based in some way.

    As it is, some C# language features will apply attributes directly, but hide them behind language syntax(fixed comes to mind, there may be a few others).

    Beyond that, I don’t really think attributes are all that elegant. They do the job but I don’t believe the syntax is even near the elegance of grammer inherent to the language.

    Chris: The answer is simple. Languages grow or die, just like everythign else. The only way to improve things is to add on to the language. There is no perfect language and believing there is and standing steadfast is usually not a solution. After years of use you can start to look at it and thing "We could do this so much easier by…" or "Performance would be so much better if…" or "Flexibility would benifit from…". Its pretty much natural. This is like asking why software developers don’t stop writing new versions of software just because the current one works.

    Also, I think you are exagerating a touch. This is nothing like C++ yet, the added complexity is pretty minor(and the benifits are extreme, why not try it first?). C++ is a massivly complex language with hundreds of litlte gotchas and confusers, like placement operators, templates(which are confusing compared to generics, albeit more flexible), etc.

  18. Darren Oakey says:

    I thought it was interesting to hear people say "stop adding features". I think to a large extent, if functionality is added in a cohesive way, then it only increases the power of the language – and most of the features they have added go towards making the language more robust, or making the code shorter, what I think is a good thing..

    I’ve also for me been tossing up the whole idea of an extensibility library, for making your own language changes. I’ve asked for some changes which probably aren’t going to happen – and been wondering whether they are worth making my own changes – I’ve thought of a very clean way of doing it… however, as pascal said above – it might be more dangerous to produce code that doesn’t look like other people’s code?

    Anyway, the main point of this message – one of our politicians years ago said something that we remember well – "there are too many laws – every time a new law is made, we should repeal an old one". I heartily agreed with the sentiment.

    So, let’s all agree, having new features added to the language, is good and necessary… the interesting question is…. what would you take OUT if you had the chance?

  19. I think this feature is very important, simply because when you use mixin techniques for building software, this is very powerfull, especially when you can override this default implementation for an interface. Just ask any ATL developer why this is a good idea!

    As for the syntax, why not take this simple example:

    class MyComponent : baseClass, IComposite = NodeComposite, IEnumerable = ForwardEnumerable

    {

    private override void IComposite.Add( IComposite child ) {

    if( this.Count > 10 )

    throw new TooManyElementsException();

    else

    NodeComposite.Add( child );

    }

    This class would take a default node implementation for a composite pattern, and at the same time I can still override some methods…

    I’ve build a complete framework this way a couple of years ago (in C++) and I can assure you this is a very powerfull technique, way more powerfull than simple inheritance.

    This technique can even solve some of the standard problems with design patterns such as the Visitor pattern.

  20. Roy Dictus says:

    Hi Eric,

    I got a request for the C# and other .Net compilers, not really a C# language request. Do you know where I can make that request official? You can find details on my blog at http://dotnetjunkies.com/WebLog/roydictus/archive/2004/06/23/17333.aspx.

    Many thanks in advance!

    Kind regards,

    Roy

  21. Anonymous says:

    Dynamically Typed » Idea 2 Language

  22. I got an email today with a suggestion for a C# language enhancement, and I pointed the person to the…