Next gen editor architecture for VS

I asked What features would you like in the next version of Visual C#? and got an interesting response back from Duncan who asks for regular expressions to be treated as an operator as the are in JavaScript and Perl.  I see there being two ways of accomplishing this.  One is to specifically add regexp support to the C# language.  I don’t really like this since it’s focusing on just one small enhancement to the language and I would prefer a more general approach that would apply to more things than just the niche of string processing.

Currently in VS we have the concept of embedding different types of code together.  Specifically ASP.Net.  For example I can do the following in the .aspx page I am authoring:


<script runat=”server”>

    public void Foo()


        Response.Write(“Whee I’m embedding C# inside html!”);



When you’re outside of the “script” tag you get all the benefits of the html editor, and when you’re inside you’re suddenly interacting with the C# editor with all the benefits that you’re used to (intellisense, colorization, goto definition, etc.).  However, this is the limit of what you can do.  Just nesting C#/VB/J# inside html.  I would like to take that concept and extend it out to any level. 

There are already cases today that would benefit from that.  The first is that VS2005 will ship with a powerful XML editor, and C# doc comments are written in XML, so why not use that editor instead for editing your comment?  If you did that then you’d get the great immediate feedback that the XML editor provides (like telling you that you can’t embed & in your comments, etc.).  Since we have a clear code marker with /// it should be possible to then host the XML editor there and have it take care of all that.  So you could then have XML Editor in C# editor in Web editor.  You could even apply that down to general purpose languages.  I would love it if when I typed:

Regex r = new Regex

We then realized you were entering the domain specific langauge of regexps and we could provide you intellisense help.  ErigGu wrote a cool tool called Regular Expression Workbench that would be cool to modify and embed as you were typeing the pattern.  That way you could test out the pattern as you were writing it and could see how the matching groups worked, etc.

Other applications would be if you were typing a new SQL query.  Or if you were doing something extremely bizarre if inside that sql query you started using xml and then were writing C# inside a CDATA block.  Can you think of other places where you’d like to see this?  Are there other tools that handle this well now?  Is it worth investigating into this, or would you just prefer that the language support regexps natively?

Comments (18)

  1. David Bossert says:

    Regular expressions? I’d prefer that you release 2005 (and .NET 2.0) before I’m old and grey.


  2. David: I’m not sure what you mean. Do you think that regular expressions are slow, or do you think that we will not get VS2005 to you because of this?

    I prefer not to think of features in those terms. It’s rather stifling to just toss ones hands in the air and say "that can never be done". If it’s a challenge, well then, those are usually the things that havent been solved and can end up being immensely helpful to people.

    There’s a set of people (Affectionality referred to as LispWeenies) who believe that all other langauges are superfluous because their power can be expressed with a subset of what lisp gives you. This is probably one of those cases as I would think lisp would be able to manipulate itself into forms that were specially designed to attack html/regexps/sql queries. Unfortunately, lisp isn’t a mainstream language today and we are left with different languages each of which optimized for a certain domain. Because of that I find the question of "how do you get the best of all worlds" rather interesting. I don’t think it’s useful to code up everything in C#, nor do i think that way about perl/C++/OCaml or any other language. However, I also think it’s too difficult to move between the different languages. Finding ways to support this sort of langauge embedding is a way (IMO) to make that far easier and having tool support for it would be a definit plus!

  3. Doug McClean says:

    Regarding my feature suggestion of custom operators, Regex could define a custom Matches operator and allow:

    string x = "Hello, world!";

    Regex r = new Regex("e.*or");

    if(x #Matches# r) {

    Console.WriteLine("It matched!");


  4. Doug: I’ve always been a fan of OCaml’s backtick notation:

    if x `matches` r

    or in C#

    if (x `matches` r)

    I think it’s clean and lightweight and allows for user defined operators quite nicely.

  5. Doug McClean says:

    Sure, that works too :). Doesn’t matter to me what the symbol is, and either one should work for the parser. (Although the backtick could be easier, since if we are allowing custom prefix unary operators there would be an ambiguity between a preprocessor directive and the operator.)

    I’ve never used OCaml, so I was shooting in the dark, but if there is a precedent for this type of thing I am all for following it.

  6. Max Battcher says:

    I have to agree Cyrus… more and more people seem to be embedding one language inside of another, and as developers it is easy enough for us to deal with the n-level recursion so it would be cool to see the tools be able to do the same… Its not "hard" to see/come up with scenarios involving RegEx in SQL in C# or C# in SQL in VB in HTML… XAML down the road yet again increases these types of recursions… You don’t even need to deal with complex integration scenarious… What about even something as simple as a "scratch buffer"? You type in /// and up pops a new temporary, memory only (perhaps with auto-save recovery), buffer in XML editor mode. Finish the XML and pop back down to your C# code, placing an "Edit as XML" smart link somewhere… You could pattern it completely off of Word’s "Smart Documents" and call it "Smart Coding". Make it pluggable and all of sudden RegEx’s can have a "Edit in RegEx Workshop" link under the standard VS "Edit as RegEx"… This would also make a useful paradigm for linking collection editors and other "magic GUI" to other types of code fragments similar to, but beyond, what is currently possible with the Forms Designer…

  7. Max: Yup, yup, yup.

    Another cool one would be if you did:

    new DateTime and we popped up a calendar control to help you pick the right date.

    I think this sort of abstract framework would be incredibly powerful and would enable so many cool things for users and 3rd parties.

  8. Speaking of embedding editors inside editors, why not take it one step further, e.g. when editing XML documentation comments:

  9. Roland: Yup. You could edit it, and then when you popped out to the C# editor you’d suddenly get this nice view of it with colors/hyperlinks/etc.

    I’m 100% for that.

  10. Duncan Godwin says:

    I’ve been thinking about this a little bit and it seems to me there are serveral different grammers that could benefit from this sort of cabability. Although I personally wouldn’t want anything as verbose as the script block above :).

    What I was thinking is a overload for the @ e.g.

    string path = @XPath"/xmldoc/customer[0]";

    This has to be one of my biggest errors whilst coding that don’t get caught at complile time. Strings that are required in a specific format and there is no way to validate until you run. Thinking about the possibilities of having these validated by the compiler; you could have XPath, XQuery, WMI SQL, T-SQL, XML, Connection Strings, and String.Format’s etc.

    Taking this one step further intellisense could be provided based on the specific grammer.

  11. Paulo Morgado says:

    * XML documentatio – It could work like regions. I could "expand" it to the XML definition or "colapse" it to a nicer reading format.

    * Special strings – I second that, XPATH, SQL, REGEX, …

  12. I like this idea, and you actually have something at MS that would help with this already.

    Compound documents. As in, when you embed an excel sheet in a word document, the sheet will store the information in a way that can say to the host "hey, I need this kind of editor, fire it up when you want to edit me". Of course, this might be difficult (or easier, if you used XML perhaps) to do if you want to preserve the text-based nature of code.

    Anders has a few ideas on this, but his outlook is different. He would like to unify it all so that you don’t have to know another language outside of the one you are working with (instead of having one document with different languages in it). I think that the workable solution in the near future is to have some sort of intelligent editing, such as is mentioned here.

  13. Nich: I happen to disagree with Anders on this. I dont’ think that we could unify this (without becoming lisp in the end).

    Roland: Just wanted to mention, but I do really like the idea of popping out and seeing this view. What’s interesting is the XML doc case is one where you’d want:

    C# -> XML -> C#

    Why? Well in your example you have the <see cref="

    In that case I’d want to have C# intellisense come up and help me pick a name right there.

    All sorts of nifty things are possible 🙂

  14. Great minds think alike–I advocated pretty much the same thing back last year:

    I see no reason why an IDE could not allow the programmer to integrate any number of languages into regions with their own Intellisense (even Word-like spell check for strings would be great) that all compile using the next-gen .NET "partial classes" feature.

    The IDE, IOW, should be language-independent even within a single file and should use some sort of method to denote regions. This could be a simple directive in the plain text of the source code file that is then rendered into collapsable, blocks, with the title of those blocks being the variable/class they define within them. Scoping of blocks could be limited (i.e., no blocks inside a method body), which would give the various compilers a reasonable set of code to work with.

  15. >> Thinking about the possibilities of having these validated by the compiler; you could have XPath, XQuery, WMI SQL, T-SQL, XML, Connection Strings, and String.Format’s etc.

    Oooh! I like that. I REALLY like that!

    Thinking simplistically, I would love extensions to C# & VB that allowed inserting literal text into code, just as ASP.NET allows inserting code into literal text. But I’d use that for things like embedding SQL and XML/HTML. If you could additionally compile validate that embedded text…well the idea just makes me all warm and tingly inside! 🙂

  16. Mike: Yup. That’s what I’m looking for it to be able to do 🙂

    Richard: There are issues.

    Say I have a C++ method, switching into VB right in the middle has some serious issue to overcome related how each treatsthe stack/heap and what objects mean to both of them.