Milestone Quality & Dogfooding (Matt Gertz)
After some hard but fruitful work, we’ve just finished our “milestone quality” (MQ) efforts, the goal of which was to get ourselves ready for developing the next versions of Visual Studio, Team System, and .NET. Although not specific to Visual Basic, I’ve opted to use this “bully pulpit” to let you know about these efforts, which certainly do impact Visual Basic along with other Visual Studio and .NET products.
As I mentioned in a previous blog post, the MQ work doesn’t involve new features or any such thing – what it does is get our tools and processes ready to go for the next cycle. It’s important to get this work done before we start coding, because it’s very disruptive to change engineering processes mid-stream. This MQ was very challenging for us, given some of the big engineering changes that we made, and I’m pleased to day that it resolved itself very well.
Like I said in that earlier post, we decided to spend a lot of MQ focusing on “dogfooding” – using our own code to get our development work done – and although we did other set-up work in MQ as well, I’m going to focus on our dogfooding efforts in this post. We dogfood our code for three reasons:
1. Improved stability for *you*: we find the bugs in day-to-day usage before the product ships to you
2. Practicing what we preach: If we say it’s good enough for you, then we should be using it too
3. It’s the right way to do things: We firmly believe that we offer the best foundation for product development, all the way from hobbyist code to enterprise solutions work. The best tool for developing Visual Studio is Visual Studio.
Now, we’re no strangers to dogfooding, but we’d never done it so early in the past. Instead, we generally made the switch during the first or second beta, after the coding was mostly done. In earlier versions (such as Visual Studio .NET 2002, where we rewrote everything) that made sense because the code really didn’t come together until that time. The downside of that was that late-start dogfooding just doesn’t give us the same coverage, because in the betas we’re winding down and focusing solely on fixing bugs instead of developing new features, and developing quality features is the whole point of VS, TS, and .NET. However, now that we’re building on a platform of stable code, we can leverage that to do early dogfooding, finding the development bugs early (or avoiding them altogether) and making adjustments to features before they become too “set in stone.” I’m really excited about being able to do this, as I’ve always envied the Windows and Office teams their ability to adopt dogfooding early.
Our dogfooding prep work this time was in two areas: Team Foundation Server and Side-by-Side support.
Team Foundation Server (TFS)
In earlier product cycles, we’d used a proprietary system for tracking bugs, another one for storing code, and some combination of bug reports and spreadsheets to track feature progress. None of these things “talked” to each other at all, and reporting status on any of them was far more complicated than it needed to be. A codebase the size of ours requires careful lifetime management handling, and it therefore has long been our goal to switch to our own product, Team Foundation Server (TFS), which handles all three (and a lot more besides).
Despite being a vocal proponent of this change, I will be very honest and say that I went into this switch to TFS with more than a bit of nervousness. Visual Studio, Team System, and .NET together comprise a really, really big division. If you just took that division (the Developer Division) alone without considering the rest of Microsoft, it would still be one of the biggest enterprise-size software vendors in the world, easily in the top five. We’ve been doing things a specific way for a long time, and that’s hard to abandon, even for more efficiency, given the number of processes that would need to change along with it.
Fortunately, these fears have turned out to be groundless. Certain of our product units (about 25%) already had pioneered TFS usage during the VS2008 timeframe, and they’d used that experience to make improvements to it. Because of their early diligence to dogfooding, the rest of the division switched over very easily a couple of months ago – we had all of our source code moved into the new project (and building!) within a couple of days, literally – and that’s a lot of code. While doing so, we identified and implemented several scaling improvements to better enable our hundreds of developers to interact with that code storage – improvements that will be ultimately benefit our customers as well.
Furthermore, we are also now tracking our bugs and features using TFS, leveraging the Team Explorer piece that ships as part of the Visual Studio family of products. This is another area where some pioneer dogfooding occurred in the previous version of VS development, so the ultimate transition for the division was also similarly smooth. The upshot of this is that we now can strong linkage between code, feature planning, and bug tracking, all with reporting services that are just far and away the best that I’ve ever seen. The savings for us (and to me, personally) in managing the lifetime of the upcoming product cycle is going to be phenomenal – I’m super-excited to be using TFS!
Side-by-side (SxS)
Of course, if we’re going to develop the next product using the product itself, we need to be able to install it as our customers do and not have it interfere with any previous versions of the product which may also be on our machines. This goal is known as “side-by-side,” or “SxS” for short. It involves a lot of re-versioning, changing registry entries, dealing with the few necessarily hard-coded strong assembly references, and so on, followed by lots and lots of testing. There are literally thousands of components that need changing between product cycles, all coming in fr om various teams. In the past, it’s something we generally didn’t complete until we reached a beta milestone, as people were hard at work writing features before beta and all of their time was very focused on that. However, deferring SxS work is not something we ever liked doing – it meant that when our customers installed our betas, there were sometimes problems lingering which required a “repair” to earlier products after the beta was uninstalled. Furthermore, as we added features in the early parts of the product cycle, we’d add that much more “SxS debt,” often resetting any SxS work that might already be going on.
So, to avoid those pitfalls, we decided to tackle this issue head-on in MQ. To be frank, it was quite a challenge to do it so quickly – there were definitely some long nights and weekends for all of us, and we ran into a lot of processes that needed retooling to accelerate our work. But by setting aside time to do this SxS work earlier, before we start coding features for the next version, not only will we be able to dogfood our own product without stomping on VS2008 installations we might already have, but your beta installation/uninstallation experience should be of much higher quality, leading to a better beta experience for you and better feedback for us.
Onward
That’s it for this post; a brief discussion on a lot of hard but good work which should ultimately lead to big wins for all of us. BTW, I’m already working on another blog post focusing on coding (which hopefully will cover something not already in covered a book this time, unlike last time J), and hope to post that up in a few days.
‘Til next time…
–Matt–*
Light
Dark
0 comments