Does GQM Work?

  • Article

I was typing a response to Matt on his blog to this post, when I realized two things.

1 - the comment forms on blogspot blow

2 - due to the length of my reply and #1 above, I thought it would be ok to respond here.

NOTE: If you don't know what GQM is and need context, read this (wikipedia), and look at Matt's previous post.

In short, the purpose of GQM is to measure something meaningful rather than the random assortment of metrics that most teams look at. Like most things in life, if you do it wrong, it doesn't work.

The OIM concept that JB responded with is hard to disagree with. It basically says "look and think about what is happening, adjust appropriately, and repeat". I don't like that the example is "Is X doing testing well" - people metrics are almost  always going to be wrong. Perhaps that is why I don't get the model...?

I'll be the first to admit that GQM is easy to get wrong, but bear with an example (and one not based on individual performance).

For GQM to work, the goal needs to be SMART (Specific, Measurable, Attainable, Relevant, and Time Bound). Let's say for example, that you're team is struggling with estimates and are trying to improve. In fact, management has asked you to improve estimates and provide data! A (smart) goal could be "Improve the accuracy of estimates for all projects by 20% in the next year compared to last year"

Questions may be results based: "What percentage of tasks planned for last year or this year were completed?", or progress based: "How accurate were my initial estimates for the last milestone?", "Did our team accomplish all of its objectives for this month?". From that, you may come up with metrics like:

  • % of tasks planned versus completed for last year versus next 12 months
  • Error of estimate for last milestone abs(actual-estimate)
  • % of tasks accomplished versus planned for last month

But wait - metrics can be gamed. In fact, it's easy to game metrics. This may be where JB's OIM model comes in. If you think your metrics will always tell you what you think they will, you are in for a big, big surprise. The answer, of course, is that you test your metrics before using them, then watch and make sure they are measuring what they think they are. Metrics aren't special - anything you follow blindly will come back to bite you.