We just shipped Vista Beta 2 and we are now on the march toward shipping what we call RCs or Release Candidates. During this time, we are forced to make many hard choices about what bugs to fix and what bugs we can live with. There are those who believe that software should ship without bugs but those people have probably never tried to ship a product of any significant size. There will be bugs in software, just like there are bugs in a new house or a new car. The important thing is to make sure there are no important bugs.
Eric Sink is a software developer at SourceGear and I recently ran across an article by him about shipping with bugs. Eric makes a few points I think worth repeating. He says there are 4 questions that must be asked:
1) How bad is its impact? (Severity)
2) How often does it happen? (Frequency)
3) How much effort is required to fix it? (Cost)
4) What is the risk of fixing it? (Risk)
The first two determine whether the bug is worth fixing at all. If the bug doesn’t pass muster there, you don’t even look at the 3rd and 4th. A bug being easy to fix is never a reason to take the fix late in a product cycle. Just because it is a single line doesn’t mean we should do it. There is risk with all fixes. I’ve seen the easiest ones go wrong.
The second two can be reasons to reject even righteous bugs. If the bug is bad and happens often but is very hard or very risky, it still might not be worth fixing. This is a hard balancing act. Sometimes the bug is bad enough that you delay the product. Other times it is possible to live with it. Making that choice is never a fun one. As owners of a product, we want all bugs to be easy to fix and have low risk. Unfortunately, that isn’t always the case and when it isn’t, hard choices have to be made.
I recommend you read the whole article. Apparently the one I found is based on a longer article which can be found here.