Agile Documentation - Tested to Destruction

Agile development is an important technique here at p&p; and throughout much of Microsoft. However, I'm yet to be convinced that it's a good way of creating user guidance and documentation. It seems to me that the process often gets in the way more than it helps to produce a great final product.

I've rambled on many times in the past about agile documentation. Most specifically in Can Writers Dance The Agile? and other posts here. Yet I keep thinking it needs deeper investigation - especially as the group I'm officially assigned to, CSI, insists we keep prodding it to see if it works.

Note: Here at Microsoft, CSI is "Content Services & International". It's probably a bit less exciting than doing clever stuff with fingerprints and DNA, but we do have fancy computers that nearly come close to those you see in the TV versions of CSI. Maybe we should call ourselves "CSI Microsoft," wear white coats, and walk around with a flashlight held over our head.

Anyway, coming back to the point, we've had another go at agile docs recently. Instead of a solid and agreed structure plan and detailed implementation notes of what we wanted to achieve, we started off with a "vision" (but see this post from a couple of weeks ago) and "brainstorming" to get a list of topics. Then we "asked the audience" with developer and advisory board surveys to see which of the topics they liked most. Next, we threw together some rough notes about each topic and then produced a first draft of each topic document.

The next stage was multiple reviews. After each review we restructured the content, and often the whole document, to get it closer to the ideal. But, because it's agile, we often changed our mind about what the doc was trying to achieve (remember, there's no detailed implementation notes to guide us here) and completely rewrote it. And then did this again after the next review pointed out the holes left by (or introduced by) the previous reworking. Some documents went through four or more complete restructurings, and several were rewritten twice.

The agile process also resulted in some of the "envisioned" topics being abandoned, often after they'd been rewritten several times, because - after investigation - they were too hard to define accurately. Or because they turned out to be not really relevant or practical. And as there is no overall structure plan, it was hard to see which topic should contain what section of the content, and how it related to other sections. It also meant that the focus could only ever be at the individual document level rather than the entire guide level, because that isn't defined yet.

But what we could be sure of was that each individual topic was precise, accurate to the nth degree, and compact with no irrelevant content. This is, of course, extremely important; especially if the content will be used as reference information. But is it still "guidance?" I guess that's the core of the problem. What exactly should "guidance" look like?

Typical equivalents of the word "guidance" include the obvious ones such as "help" and "advice." However, there are broadly two categories of meaning: "leadership" and "assistance." These almost seem like opposites - one leading from the front and the other pushing from the back. Yet the sub-meanings according to my thesaurus include "direction", "support", "management", and "control." Some of these seem more like they are related to aiding through understanding, whereas others are more related to despotic regulation. I'm going to take a guess that we're aiming for the first of these.

So did we end up with what we wanted, and does it aid through understanding? It's not completely finished yet and it will be a while before we see any user feedback. And there's no doubt that the content will be extremely useful for the specific users and use cases it addresses. But it still seems like we missed opportunities. The agile process narrowed the focus and transformed the content based on individual reviews of segments, and forced additional depth of detail. It also removed a lot of the general "understanding" content because it already was familiar to the experienced reviewers. Most of all, it resulted in huge amounts of extra work writing and repeatedly updating (and then sometimes discarding) the content.

Without a predefined (if flexible) structure and an overall feel for how it will all fit together and appear to the readers, there is nothing to prevent this wandering. As a writer, I'm lost if I can't see the finished thing in the back of my mind. It's like driving through a city in your car while blindfolded, and navigating by reversing and choosing a new direction at random every time you hit something.

Maybe agile is good thing that can help to focus guidance more accurately. Or maybe it just allows the guidance to wander away from the original vision and risk irrelevance. If the original plan was to provide guidance around X and you end up with fabulous documentation of Y instead, did you do a good job?