Test-Driven Guidance

When I last met with Rob Caron to walk him through Guidance Explorer, one of the concepts that peaked his interest was test-cases for content.   He suggested I blog it, since it's not common practice and could benefit others.  I agreed.

If you're an author or a reviewer, this technique may help you.  You can create explicit test-cases for the content.  Simply put, these are the "tests for success" for a given piece of content.  Here's an example of a few test cases for a guideline:

Test Cases for Guidelines

Title

  • Does the title clearly state the action to take?
  • Does the title start with an action word (eg. Do something, Avoid something)?

Applies To

  • Do you list technology and version? (e.g. ASP.NET 2.0)

What to Do

  • Do you state the action to take?
  • Do you avoid stating more than the action to take?

Why

  • Do you provide enough information for the user to make a decision?
  • Do you state the negative consequences of not following this guideline?

When

  • Do you state when the guideline is applicable?
  • Do you state when not to use this guideline?

How

  • Do you state enough information to take action?
  • Do you provide explicit steps that are repeatable?

Problem Example

  • Do you show a real world example of the problem from experience?
  • If there are variations of the problem, do you show the most common?
  • If this is an implementation guideline, do you show code?

Solution Example

  • Does the example show the resulting solution if the problem example is fixed?
  • If this is a design guideline is the example illustrated with images and text?
  • If this is an implementation guideline is the example in code?

Additional Resources

  • Are the links from trusted sites?
  • Are the links correct in context of the guideline?

Related Items

  • Are the correct items linked in the context of the guideline?

Additional Tests to Consider When Writing a Guideline

  • Does the title clearly state the action to take?
  • Does the title start with an action word (eg. Do something, Avoid something)?
  • If the item is a MUST, meaning it is prevelant and high impact, is Priority = p1?
  • If the item is a SHOULD, meaning it has less impact or is only applicable in narrower circumstances, is Priority = p2?
  • If the item is a COULD, meaning it is nice to know about but isn't highly prevelant or impactful, is Priority = p3?
  • If this item will have cascading impact on application design, is Type = Design?
  • If this item should be followed just before deployment, is concerned with configuration details or runtime behavior, is Type = Deployment?
  • If this item is still in progress or not fully reviewed, is Status = Beta?

Benefits to Authors and Reviewers
The test-cases serve as checkpoints that help both authors and reviewers produce more effective guidance.  While you probably implicitly ask many of these questions, making them explicit makes them a repeatable practice for yourself or others.  I've found questions to be the best encapsulation of the test because they set the right frame of mind.  If you're an author, you can start writing guidance by addressing the questions.  If you're a reviewer, you can efficiently check for the most critical pieces of information.  How much developer guidance exists that does not answer the why or when?  Too much.  As I sift through the guidance I've produced over the years, I can't believe how many times I've missed making the why or when explicit.

I'm a fan of the test-driven approach to guidance and here's my top reasons why:

  • I can tune the guidance across a team.  As I see patterns of problems in the quality, I can weed it out by making an explicit test case.
  • I can tailor test cases based on usage scenarios.  For example, in order to use our checklist items for tooling scenarios, our problem and solution examples need to have certain traits.  I can burn this into the test cases.
  • I can bound the information.  When is it done and what does "good enough" look like?  The test case sets a bar for the information.
  • I can improve the precision and accuracy of the information.  By precision, I mean filter out everything that's not relevant.  When it comes to technical information to do my job, I'm a fan of density (lots of useful information per square inch of text).  Verbosity is for story time.

Examples of Test Cases for Guidance
I've posted examples of our test-cases for guidance on Channel 9.