In software development, there is the “triangle” that teams have to navigate through while shipping a product – the triangle of quality, features and time to market. There are always trade-offs to make but in my mind if push comes to shove quality wins hand over fist. I always tell people that 2 years from now, nobody (well almost nobody) will remember the exact date we shipped a product, but people will absolutely remember whether it was a quality product or not. At the same time, I also understand the need for us to be more predictable in software delivery because our customers and partners rely on us and take a bet on us delivering our technologies and products in a predictable schedule.
We originally planned for delivering Beta 2 by the end of March 2005. However, we ended up holding Beta 2 for a few weeks because we wanted to get to the right level of quality for this Beta 2 release to enable go-live license and production deployments of applications built on Visual Studio 2005 Beta2. We had very high stress goals across the various components. Stress tests are primarily designed to time-compress and simulate customer workloads into a relatively shorter amount of period to ensure that the system can withstand what you would see typically in a customer situation over a long period of time. We wanted to stay firm on meeting the stress goals to ensure that this is the highest quality beta that we have ever shipped.
We also did something for the first time with Beta 2 where we involved a number of our TAP (Technology Adoption Partner or early adopter) customers to actually sign-off on Beta 2 before we would release. This was quite a valuable exercise. We provided early drops of Beta 2 to these customers who gave us valuable feedback. We actually ended up finding and fixing a couple of issues and finally we got 100% sign-off from the TAP customers who participated in this. Our plan for RTM is to absolutely have an expanded set of TAP customers to sign-off on the product before we ship.
I know that we are a couple of weeks behind where we wanted to be. But I feel really good that we held Beta 2 for a little longer for the right quality reasons.