When specs go bad


If your team writes specs before starting to code (as opposed to after, when it’s easy to do so, and the spec is more commonly called documentation) you certainly have cause to be happy, but you aren’t home free. Another bone of contention awaits: whether a particular bug is a code bug or a spec bug.

Sometimes a program will function correctly per the spec but that functionality will be determined to be incorrect. More often, the spec will be hazy about or outright omit details regarding a certain edge condition, unusual circumstance, or completely mainstream scenario. The developer may not see this omission as a big deal and so will decide on her own what should happen. If you’re lucky, the developer chooses the same behavior as you (and your customer) expect. If not, you are in spec bug limbo.

Your best course of action at this point is to discuss this issue with your feature team (or, at the very least, your developer). As with most everything else in this industry, a little conversation often goes a long way, and a reasonable resolution is shortly reached.  If the question turns out to cause massive controversy, well, at least y’all’re talking it through, which is far better than the alternative:  embarking on a Bug Bounce.

“Bug Bounce” is my term for a fight between team members that is carried out via the bug database. It usually goes something like this:

  1. Tester finds some odd behavior.
  2. Tester logs a bug and assigns it to Developer.
  3. Developer begs to differ and resolves the bug back to Tester as Not A Bug.
  4. Tester reactivates the bug back to Developer, perhaps adding more detail.
  5. Developer re-resolves the bug back to Tester.
  6. The bug continues to bounce back and forth until a) somebody realizes this is really a spec bug and assigns it to PM, b) Developer and Tester talk to each other in person and work it out, or c) Developer and/or Tester become completely fed up and take their frustration out on each other — never a pretty sight (although sometimes a hugely entertaining one <g/>).

Not every disagreement between Dev and Test is due to a spec bug, of course. A statement that is completely clear to one person may be rife with ambiguity for another.  Whether this is a spec bug is debatable. The effort required to make a spec completely unambiguous to all parties is rarely worth spending, so some amount of ongoing discussion is to be expected. Just remember it’s all fun and games until someone gets their code poked out.

*** Comments, questions, feedback?  Contact me at michhu at microsoft dot com.

Comments (3)

  1. Andy Johns says:

    Here’s one that happened to me not too long ago, spec item 2.3.i clearly specs out feature X. Spec item 5.3.ii contradicts item 2.3.i, but not in way that is uncodable….. System is developed, rolled out. Testing is organized by spec section. Tester for section 2 reports bug in feature X that is due to spec item 5.3.ii. Tester for section 5 reports bug in feature X that is due to spec item 2.3.i. The developer throws both bug reports back, and is caught in a double bounce between the two testers….

  2. AndrewSeven says:

    "Not a bug" shouldn’t be sent back to the tester.

    If you can’t reproduce the behavior, send it back to the tester with "Need steps to reporduce".

    If you can reproduce the behavior and it is not a bug, send to the PM and let the PM decide what the path towards resolution is.

    This of course assumes that the PM actualy manages the project.

  3. Take Outs for 3 March 2004