to CTP or not to CTP

  • Article

We just recently released the June 2005 Community Tech Preview (CTP) for VSTS. It’s been an interesting process because this is the first CTP for VSTS where we simply picked up one of our daily builds, made sure it passed some build verification tests (BVTs) and cut it on to a DVD. All previous VSTS CTPs were the result of weeks of testing and stabilizing for the purposes of refreshing our dogfood system. As part of that we would fix blocking bugs, validate the installation guide, have a few folks that had never installed VSTS try it unassisted to make sure the instructions were clear, update the known issues document to help point out the land mines, etc, etc. 

For this release we did none of that. There are some among us that believe that without all this stabilization that we shouldn’t release our bits. There are others that say that providing timely and predictable snapshots of our current state is helpful to our customers because they can see what we’re doing and provide us timely feedback. They might even see that their bug has been recently fixed. 

Of course, we’re all just supposing now since we don’t have your input. So, I consider this June CTP as a bit of an experiment to see what value these frequent builds of VSTS can be to our community. I certainly wouldn’t recommend moving any active development to this build but maybe you’re like my buddy Etienne who likes picking up the latest and greatest just to sniff it out and see how we’re doing. He sends us good feedback even on daily builds and perhaps you will too. 

It seems to me it's all about setting expectations. If we can, ahead of time, tell you what we've done to evaluate the bits before we released them (run BVTs, using internally, full test pass, etc), then you have enough information to make intelligent choices about which build to download. Also along this line of thinking, one thing that we’re discussing is improving the ability for you to rate our releases so that there can be a reliable feedback loop visible to all. The thought is that if we had a number of recent builds to pick from, providing a report of user ratings would help everyone navigate to find the build that has the right level of freshness and known stability for their purposes.

I’m really curious what your thoughts are on this whole CTP thing so please let me hear from you.

Thanks,
jeff