Recall class
So in our drive to get beta2 out the door we've been continuously raising the bar on what sort of changes are allowed to the codebase. As you can well imagine there is risk with pretty much any change and at a certain point the need to keep the product stable outweighs the benefit of a fix. Even the most innocuous fix can end up causing problems in ways that you never realized (especially in a huge nondeterministic product like VS with all the threads that it contains).
As of this week we raised the bar to what we call "recall class". Specifically, a bug fix needs to be for something so bad that we would recall the shipping of the product with all the enormous cost involved in that. Why is that cost enormous? Well, not only does it mean recalling the media we've printed, throwing it all away, and reprinting it all over again, but it also means that we've noe delayed all the companies out there that want to use beta2, and we've potentially caused them to have to delay their own releases. It cascades into something pretty horrendous and you want to avoid it at all costs if possible.
So what does it take for a bug to be "recall class" worthy? Well, that's tough and invariably comes down to a case by case investigation of each issue that arises. A good rule of thumb IMO would be "siginificantly affects a large number of customers in a negative way in an highly common scenario, and which there is no way for the customer to work around the issue." To break it down a little, i would expect a recall class bug to have the following characteristics:
- affects many customers. If only 1 out of 10,000 customers is affected by a bug, well, then it's something that we can wait to fix
- is a significant problem. If we have a small colorization bug, then we don't care if it's affecting 99% of users since they can continue to be productive in the face of it.
- you can't work around it. Enough said. If there is something quite simple that the user can do to get aroudn this, then we can just add this to the "Readme" and still get it out there.
So, for example, the runtime not loading any dlls without dying no matter what... that would be bad. That's an easy call.
The C# compiler producing unverifiable code in very common code... that would be bad. Now, if you could make a small change (say a command line switch), to get around this, then it probably wouldn't be "recall class". *but* if it required the user to work around the problem anywhere they used this "common code" then that would probably make it "recall class".
Now, where it gets fuzzy is when it's not clear if all the above criteria is met. For example, take the following hypothetical bug:
"When typing the switch keyword there is a high probably of a hang in the IDE" i.e. if you have:
class C {
void Foo() {
switch
void Bar() {
}
}
then VS completely hangs.
Well, it would certainly affect many customers. The entire C# user base would be affected by this as it would make typing a common construct incredibly risky. It would certainly be significant. Hangs are up there with crashes and data-loss as being the worst bugs in the product. Even though we have automatic backups (every few minutes or so), it means that your current work is lost not to mention the time wasted killing/restarting VS and figuring out what you need to redo. Now... you can work around it. You can avoid the "switch" construct, or you can do something like the following:
Start by typing in the body of the switch like so:
class C {
void Foo() {
{
}
void Bar() {
}
}
Then add the switch keyword
class C {
void Foo() {
switch
{
}
void Bar() {
}
}
and you'll be fine.
So is this recall class? It's just not clear. You could work around it... except that it's not clear if users will figure out how to work around the problem. The workaround is also not simple. Even if you figure out how to not trigger the bug and you're savvy enough to use the workaround, then you could still be hosed a few minutes later when you forget and just type "switch". So, one could argue that the workaround isn't actually simple. Since it's not jsut a one time fix that prevents the problem entirely, but rather is a complete change to how a user would normally edit code and which the user would have to vigilantly enforce over themselves, it could probably be considered bad enough.
My desperate hope is that the C# IDE doesn't have anything as bad as that show up while we're doing tons of testing over beta2. I just want this out the door and in your hands. Now, it just remains to be seen if our code is good enough.