Learning the new editor

I had a professor in college, Sidney Marshall (also known as "the man, the myth, the legend: Sidney Marshall"), who was and still is a pretty big inspiration to me. He's a fairly big Lisp guy (part of what got me interested in Lisp in the first place), quotes various pieces of the ISO C++ spec conversationally, and is smarter than about every other person I've met. He one of those people who really does know a little bit about everything, too.  Whenever you hear him start a sentence with “You know what the real problem with blah is, right?” at 9 in the morning, don’t assume you’ll be on-time for your 10am class.

In one of our conversations, he made a rather interesting point that has stuck with me. I'm not sure what the original topic was, but he was explaining something along the lines of why it's so easy for him to write a lot of the cool stuff that he does. As he explained it, the biggest difference between him and a young kid like myself is that the time it takes for him to design something is near zero. When I try to solve a problem, I have to think about it or prototype around it for awhile, due to my inexperience with about everything. For him, however, he's been doing so much for so long that he can skip that step entirely and just start in on the solution.

I never really fully grasped the meaning of that until somewhat recently, when I started writing and helping others write extensions for the new editor. After working on the editor team for about 2 years, I know most of the editor pretty well (with the exception of IntelliSense, which I've never had the occasion to use, extend, or really take a look at). However, for almost everyone who didn't write the editor, it's an entirely new API, a different set of concepts from the previous Visual Studio editor, and even a few things that may be new or interesting in general for extending an editor.

I’ve found that I’m starting to understand what it is Sidney meant – now, when I sit down to write an editor extension, I don’t really spend any time in the design phase, which is an interesting way to write code :)

The GoToDef extension

To give you an idea of how I write editor extensions, I'll walk through the design of the GoToDef extension I recently wrote (partially modeled after an earlier version of it, written by Huizhong, a member of our excellent QA team).

A screenshot of the GoToDef extension, from the Visual Studio Gallery

Here's what I set out wanting the extension to do:

• Ctrl+clicking on an identifier executes GoToDefn in Visual Studio.
• Ctrl+mouse over on an identifier (hopefully only identifiers) turns the identifier into a html-esque link, so that you know what you can click on.

To start with, here are the major pieces of my component, the first 4 of which are exported to the editor via MEF:

1. A (view-specific) classifier for turning things into links. It's told which span of text should be underlined/blue.
2. A classification type for the underline, along with a format that sets the foreground color to blue and adds TextDecorations.Underline to the text.
3. A keyboard processor, for tracking when the ctrl key is held down and released. This isn't entirely sufficient, as you could press/release ctrl outside of the editor, but it's an important part of tracking that behavior.
4. A mouse handler that handles (a) mouse move, for updating the classifier on what should be underlined (if anything), and (b) mouse left-button up, for sending the command.
5. There's also a small, non-editor piece called CtrlKeyState, which is updated by the mouse and keyboard processor when they see the state has changed and events accordingly (so the mouse processor can do it's magic to determine if the word should be highlighted or not.

(Though it may sound like a lot, the whole thing is only a couple of hundred lines long). Then, I moved on to following services to use to make this job easier, from what the editor provides:

1. A classifier aggregator, to consume classifications that are provided by other extensions (like the C# and C++ language service). As it turns out, both those language services (and possibly others, though not VB, unfortunately) colorize/classify identifiers and things like User Types, which the mouse processor consumes to guess what things GoToDefn will succeed on. Ideally, we'd just be able to ask whichever language service if the identifier under the cursor can have GoToDefn executed on it, but it appears that QueryStatus for that command always succeeds, and whoever handles it on the language service side just pops up a little dialog box telling you it fails when you shouldn't have clicked it.
2. A text structure navigator, for determining the extent of the current word under the cursor.
3. An IVsTextEditorAdaptersFactoryService (whew, long name), which can be used to get, for example, an IVsTextBuffer from an ITextBuffer, which could be then used to get a service provider and find the shell service for executing global commands (since this gets into the ol' COM way of doing things, it's kinda ugly to write in managed code, and I mostly just used the code Huizhong wrote for that. Kudos to her for figuring it out :). Thankfully, post beta 1, you'll just be able to [Import] an IServiceProvider directly, and skip all those hoops you have to jump now.

It only took me a couple of hours to write the extension (as our PMs say, “it’s only code”; as I say, “I’m going to punch you”), with a little bit of that time spent getting over small hiccups along the way.

My point, though, is that once you understand the API of the editor, it doesn’t take long to create what it is you want.  Put alternatively, I don’t think there is much overhead imposed by the editor – the hardest part of writing an editor extension is the logic specific to the extension’s purpose and not the the logic that has to “deal” with the editor.

So the question becomes: how do you learn the editor, without having written it?

Editor extension, in brief

Oh, the irony.  Nothing I write is ever brief.

I'm not sure whether it was a conscious decision in all cases or not, but extension points in the editor tend to have a few characteristics:

Simplicity - Most extension points are very simple and have a very minimal API. Classifiers and taggers, for example, have only one method (for getting classification and tag spans) and event (for informing consumers that classifications or tags have changed) apiece. The contract is simple enough to comprehend pretty quickly.

Single purpose - Extension points tend to do only one specific thing. In the past, if you wanted to provide language specific features, you'd write a language service (which is still supported, by the way). Using the new extensibility mechanisms, however, you'd extend the features you want. You (or anyone else, for that matter) can write a classifier to provide colorization for any language, without regard to who else is providing other features around that language. Moreover, you can write a classifier for a language that already has a classifier written for it - the editor figures out how to combine your contributions.

Follow common patterns - Common API for common scenarios - tagging is probably the best example of this (which I say without any hint of bias, as I wrote most of it - as such, send your complaints my way :). If you want to provide, say, outlining regions, error squiggles, text marker highlights, classification (you can do it with either a tagger or classifier), or certain types of adornments in the text, you'll write a tagger to do so. All these things share a common trait - some component at the level of the buffer wants to provide information for others to consume, oftentimes at the visual level. It's pretty handy to separate these out, so that:

• The component that knows about errors in your language doesn't also have to know how to draw a red squiggly line in the editor
• If someone else wants to visualize errors differently (like by placing a little red line in the vertical scrollbar), they can do that without caring what errors are in the underlying language, or even what the underlining language is (an error is an error, in the most general sense).
• If a new language comes along and wants to provide errors (or someone wants to provide squiggles over something else in an existing language), they can do so without worrying about how to visualize it (a fun little extension to write: put squiggles underneath every instance of "TODO" or "UNDONE").
• The tags provided also go through an ITagAggregator, which takes care of things like combining tags from multiple taggers on the same buffer, and combining all the tags from different buffers in the same buffer graph. This makes it so you don't have to worry about other extensions and whether or not projection is involved - the real beauty of projection buffers is that you (almost) never need to know about them.

So we have the tagging subsystem, which unifies the API for all of these. You can also use it for simpler stuff (say you just have two components that want to communicate from the buffer level to the UI level), and you can define your own ITag types, and thus create new extension opportunities.

In general, then, editor extensibility can be thought of as being wide and not deep. That's not to say that the extensibility doesn't permeate fairly completely, but that you don't need to dig very much to hit water.

The hard part, then, is getting people who want to extend the editor up to speed on the breadth of extensions they can provide and the services they can consume.

How to learn the editor

Between my blog and the official editor team blog, there will be more articles upcoming on tackling this from a few different angles.

To start, I’m planning to write some articles explaining the extensions I wrote (and any more that I end up writing) and providing code samples.  This seems to be a common way of learning a new API, so I assume it will be helpful for many people.

At some point, Jack will be writing some articles on the team blog about the pieces of the editor and how they all fit together.  I’ll probably write a few posts on my blog about some things (Tagging, for sure; the differencing stuff, since I wrote it; writing classifiers/taggers and best practices around that; probably a description of how outlining works in the new editor; etc.).  If there is a specific topic you want to see covered, add it in the comments below.

There is some documentation on editor extensions and the like, with a decent overview of the common extension points for people to use (I’m not sure if there is one for various services that are provided).

Comments always welcome.