I'd been running test builds of Windows Vista for over a year prior to its release last week, and I wasn't alone: thousands of Microsoft employees were testing Vista to iron out the kinks. Our Partners in the Early Adoption programs also slug their way through Community Tech Previews and Betas. Some of us internally were additionally running less polished weekly or daily builds.
The process was sometimes fun, but not always. The late-night question "hon, what are you doing on that computer?" soon morphed into "hon, are you installing Vista again?"
But the feedback loop from the Vista team was impressively tight. Our questions, complaints (and compliments) were addressed swiftly. And I wanted to believe that all of our testing helped bring us to Jim's signal that "it's time."
I got a really nice mail the other day, which was sent to everyone involved in Vista reliability testing. It pointed us to the Vista Release to Manufacturer Press Pass, and it read:
"FYI - you were part of this (see highlighted below). Thank you!!"
PressPass: How do you define quality in relation to an operating system release? What are the important elements?
Hallauer: It’s a combination of the core operating system capability, plus ecosystem support and compatibility. In terms of core quality, we talk about three important factors — reliability, usability and security.
Reliability. We didn’t just fix the bugs, we took a new look at the classic places where customers have had the most pain. For example, we have a much more robust feedback mechanism built into Windows Vista to detect application crashes and hangs. We made a special effort to fix those issues, both through bug fixing and architectural changes in Windows Vista.
We also built mechanisms into the product to get more actionable feedback from customers to understand where their pain points are. Furthermore, we made that same degree of feedback available to our partners, and we made it easier for them to put updates on Windows Update, so that changes can be propagated to customers more easily. Finally, with Windows Vista we also doubled the number of stress tests that we run on a daily basis compared with Windows XP SP2, including a new kind termed “long haul” stress testing where systems are monitored over a 15-day period to ensure that they don’t exhibit any reliability issues. So there are a large number of factors that contribute to improved reliability in Windows Vista.
I was really grateful to receive the note - even if my contribution is like a tiny drop in an oceanic vista - and I look forward to seeing the results in the operating system's reliability over the "long haul."