Programming in the Blue


It amazes me how difficult it is to design new features to a programming language.  Once a language is baked in its original form it’s almost impossible to add new semantics let alone syntax rules.  Operators are hellish.  You want to invent new ones, but you are generally limited to the common symbols found on the standard keyboard.  That’s why so many languages focus on using braces and brackets, because they are free and easy to type.  Though, once you’ve allocated them to one purpose you cannot readily reallocate them to another.  You can try to overload them, and give them some contextual dependency, but that usually ends up in a mess.  I know, I advocated for a long time using the brackets as a filter operator for the X# language.  I even built in all the contextual baggage into the compiler so it would fundamentally work, most of the time.  In the end it was a wash.  If I could do it all over again, I’d find another way.


 


We get into this same trouble when designing new features for C#.  Symbols to use as new operators are scarce.  It’s even worse with keywords, you cannot make new ones.  That would break backward compatibility of the code.  Sure, C# 2.0 pulls some tricks and lets in some new terms in novel places, but you can only do this so much before that avenue dries up.  We bat around ideas about how to solve this problem once and for all.  Sometimes it seems like we spend way too much time thinking about it.  But, you know, eventually hard work does pay off.  In fact, I’ve come up with a solution that should suffice for all .Net languages.  I’m not certain, but it might be applicable to other languages as well.  Fortunately, for Microsoft, I’ve already filed the patent on it. It’s been a while, but legal has finally given me the go ahead to fully disclose it now.


 


You see the trouble really lies in the in-ability to set pieces of the language apart.  What you want to do is have a mechanism you can use to specify that a term is always special to the language, so that it can never be used by an application as a variable name or some other identifier.  Doing that solves the backward code compatibility problem. However, there is still only a small set of common symbols on the keyboard, so it’s not easy to find one.  Sure, you could imagine using other symbols available in the Unicode character set, but then you’d have to invent new keyboards to use just for the language, and I don’t want to re-invent the APL nightmare.  Still, if you had a spare symbol or two you could come up with a bracketing mechanism to set your keywords apart.  Some languages do this with a dollar-sign ($) prefix, other languages have other means.  But this is an ugly solution.  Extraneous symbols just get in the way of readability, and if you know anything about maintainability of code, you don’t want anything hindering you ability to read and understand it.


 


So I found myself perplexed one day last month, wanting to solve the problem but having no luck.  I’d tried just about every combination of symbols that I could think of, and yes I even tried putting keywords inside quotes, but it just looked bad.  I would try each one out using the IDE and put it in context with fragments of code.  Anders does this a lot when he’s working through a new idea, and it always seems to work for him.  I try to learn new tricks like this whenever I can.  Still it wasn’t helping.  Have you every tried to seriously read code that wrapped keywords in something like dots?  Try it


 


.foreach.(int x .in. numbers) …


 


Now tell me, is that a usable solution?  And that was the best one.  I just could not bear to end up with a language that looked like FORTRAN, could you?


 


So as I said, I was perplexed.  I was intellectually exhausted at this point.  It had been weeks and still nothing.  But, you know, that’s exactly the time when breakthroughs happen.  I was weary and almost to the point of beating my forehead into the monitor, when suddenly I saw something.  It was less of a thought and more of an optical illusion, a trick of the light, like when you eyes lock onto one of those mystery-eye puzzles, and then you truly see what has been there all along.  It hit me so fast I found I could not breathe.  The answer had been right in front of me the whole time.


 


I had been staring at a fragment of normal looking code, ready to be cut-and-pasted into yet another bad idea.


 


public class Foo {


  public int X;


  public float Y;


}


 


The answer was so simple it shook me to the bones.  We did not need new symbols.  Symbols got in the way.  We had something so powerful already that not only got out of the way, it enhanced readability. 


 


I’m talking about the blue.  Keywords are blue.  Everyone already knew this, but no one had ever seen it for what it truly was, for what potential it truly had.  It was the big jump that made the IDE experience so powerful.  Code was easier to read because of it, and it didn’t get in the way of reading the words.


 


Text has color.  Color has meaning.  Keywords are blue.  Believe me, from now on programming will never be the same. 


 


Sure it sucks for people using notepad or third party editors, but you cannot stand in the way of innovation!


 


Keep programming in the blue.


 


Matt

Comments (17)

  1. Me says:

    So how does this change the source code byte stream?

  2. Matt says:

    This is actually not too hard. All you have to do is set to blue bit on for each byte of the keywoard. The blue bit is the 9th bit from the left. Of course, we need to do a bit of redesign of the microprocessor to make it so, but in the end we’ll have a much better language.

  3. Androidi says:

    Hahha, what’s next? Having different colors for same keyword to separate the functionality? Like green for the using directive while blue for the using statement 😉

    By the way, i’ve seen yellow used in the Whidbey demos as a background color for text, do you know are these colors hardcoded?

  4. Richard Roberts says:

    You said "You want to invent new ones, but you are generally limited to the common symbols found on the standard keyboard." . What kind of operators are you thinking of? Once you get past overloading operators for +,- etc and bitwise comparisons, I struggle to think of where new operators would be helpful. Surely the use of a method would do the job just as well, and more intuatively.

  5. Jonathan Perret says:

    Uh. I thought you were serious, but the other comments show that I’m the only one who did not get the joke (shows I haven’t been reading your blog for long, I guess).

    Anyway, I was about to say that this was not such a bad idea : why not have keywords be actually (in text) delimited by dots or whatever, but displayed without the dots and in color. Well, whatever.

    The other thing I was going to say is that Chuck Moore can actually claim prior art : http://www.colorforth.com/cf.html

    Cheers,

    –Jonathan

  6. Mark M says:

    I guess those of us that are colorblind need to move into program management ASAP!

  7. Matt says:

    I am humbled by the sheer elegance of color forth. The code is stored binary, in huffman coded blocks. It doesn’t have the blue-bit, but it has bits for red, green and yellow. Now that’s worth programming backwards for!

  8. John Roth says:

    I see someone else has mentioned Chuck Moore’s Color Forth, so I’ll just note it in passing.

    The other problems are relatively easy to solve as long as you take the step of making the editor/IDE do some work in displaying the program in a way that’s useful to the programmer.

    You’ve already mentioned the fix for keywords: just put a special character in front of them and let the editor/IDE sort it out for the developer.

    The problem with operators goes back to C’s "maximum munch" rule. If we divide special characters into two groups: those that will always and forever be singluar and never occur in multi-character operators (the parenthesis, brackets and braces of the world), and all others, then the lexer should simply collect the string until it runs into a character not in the set, and look it up in the dictionary. If it’s not in the dictionary, tough. Its an error in this release.

    The other problem with operators is that most of the operators I’d like to see are in Unicode, so all that needs to be done is have the editor/IDE provide a key sequence to specify it.

    None of this is rocket science, it just requires stepping out of the "but I want to write programs in Notepad" whine.

    John Rooth

  9. The Younger Brother says:

    What do you do if there is blue text within quotes?

    Man. I wasted way too much of my lunch break reading this thinking there was something to it.

    Be afraid.. be very afraid. I know where you live!

  10. You’ve been Taken Out! Thanks for the good post.

  11. Jonathan Perret says:

    John Rooth : […]most of the operators I’d like to see are in Unicode[…]

    Shades of APL, eh ?

    http://en.wikipedia.org/wiki/APL_programming_language

    I seem to remember the C64’s BASIC environment had a use for the funky characters found on the keyboard; there’s also the precedent of MPW, of course (http://developer.apple.com/tools/mpw-tools/commandref/appc_misc.html).

    Perl, on the other hand, does not have them but positively _screams_ for Unicode operators.

    Cheers,

    –Jonathan

  12. Rick Byers <rick@kinitos.com> says:

    I think this post, and some of the comments, are touching on a fundamental limitation to most current programming environments.

    Think about it for a second: we invented programming languages as an abstraction layer between human thought and CPU instructions, with a compiler doing the transformation. .NET (and many other environments) added another abstraction layer between the language and the CPU, with a managed execution environment doing the transformation. I believe computing is all about designing abstraction layers like this to manage complexity. You can come up with a bunch more layers in the chain from human thought to electrons – and each one adds value and reduces complexity. So why do we usually treat the output from people as the direct input to compilers? Why not introduce another abstraction layer with the editor doing the transformation between what people can easily write/read and what compilers can easily process?

    For example, instead of storing C# code on disk, we could store the parse tree (or an analog to it) in XML form. Then, the visual studio editor does the work of converting to/from C# on the fly. The parse tree more accurately captures the semantics the user was trying to embody when they typed the code. If you want to add a keyword to the language, you just have the compiler support a new element type. The editor then unambiguously displays this to the user in whichever way the user finds most intuitive (font colour/style if you like). Old code is not affected because it has tags like <identifier name=”newKeyword”/>.

    Another problem solved by this approach is the “using” directive. Writing “using System;” is purely for human benefit, and just makes the compiler’s job harder. Specifically, as soon as a developer types a non-namespace qualified class name, the precise semantics of which class he meant is potentially lost forever. Sure today it was unambiguous, but once some libraries are updated, that identifier may become ambiguous. Namespace qualification is something that should be a user preference. Just because the writer is experienced with the class libraries enough to know which classes his identifiers refer to, doesn’t mean the next person (or compiler) reading his code will. Why not capture the developers full intent in the file saved on disk, but just hide the details from users that don’t want to see it.

    The list of benefits this approach could provide goes on and on (refactoring, searching, custom language extensions, static analysis, etc.), but I’ve already ranted too much. Of course there are some real challenges with this approach (“structured editors” have been tried before, and have generally failed due to user-interaction problems), but I am confident that someone with the resources and experiences like Microsoft could overcome the problems and massively improve developer productivity as a result.