to CTP or not to CTP


We just recently released the June 2005 Community Tech Preview (CTP) for VSTS.  It’s been an interesting process because this is the first CTP for VSTS where we simply picked up one of our daily builds, made sure it passed some build verification tests (BVTs) and cut it on to a DVD. All previous VSTS CTPs were the result of weeks of testing and stabilizing for the purposes of refreshing our dogfood system.  As part of that we would fix blocking bugs, validate the installation guide, have a few folks that had never installed VSTS try it unassisted to make sure the instructions were clear, update the known issues document to help point out the land mines, etc, etc. 


For this release we did none of that.  There are some among us that believe that without all this stabilization that we shouldn’t release our bits.  There are others that say that providing timely and predictable snapshots of our current state is helpful to our customers because they can see what we’re doing and provide us timely feedback.  They might even see that their bug has been recently fixed. 


Of course, we’re all just supposing now since we don’t have your input.  So, I consider this June CTP as a bit of an experiment to see what value these frequent builds of VSTS can be to our community.  I certainly wouldn’t recommend moving any active development to this build but maybe you’re like my buddy Etienne who likes picking up the latest and greatest just to sniff it out and see how we’re doing.  He sends us good feedback even on daily builds and perhaps you will too. 


It seems to me it’s all about setting expectations.  If we can, ahead of time, tell you what we’ve done to evaluate the bits before we released them (run BVTs, using internally, full test pass, etc), then you have enough information to make intelligent choices about which build to download.  Also along this line of thinking, one thing that we’re discussing is improving the ability for you to rate our releases so that there can be a reliable feedback loop visible to all.  The thought is that if we had a number of recent builds to pick from, providing a report of user ratings would help everyone navigate to find the build that has the right level of freshness and known stability for their purposes.


I’m really curious what your thoughts are on this whole CTP thing so please let me hear from you.


Thanks,
jeff


Comments (8)

  1. I can’t speak via the VSTS, but the professional version won’t even

    install properly on a clean machine. (SQL Express fails with a

    dependency on .NET) I had to pull up an image with Beta 2, remove *most*

    of beta 2 (leaving SQL Express & VSS), and then install the June CTP

    just to get it to install.

    I think there should be a minimum level of quality, or a good list of

    gotchas to make it easier to test things.

  2. Glenn Berry says:

    I would rather that the VS2005 and SQL Server 2005 CTP’s stay in sync, i.e. the June or August CTP work with each other, use the same version of the 2.0 Framework, etc.

    You guys (VS and SQL Teams) need to go through some practice evolutions of releasing compatible versions before you go RTM. Just my two cents…

  3. As I mentioned before, we’re in the process of updating our internal version of VSTS with updated bits. …

  4. Carl Franklin has posted the .NET Rocks Show #117 with members of the Team System team ⊕

    Chris…

  5. Joku says:

    It would be best for all if, instead of literally blogging "Do not install CTP X cause it sucks", you would choose to use widely understood terms. Just an example, to give some idea.

    So to clarify:

    Nothing

    < VSTS DB 2005-06-15 "download/remote access only to see if your bug is fixed"

    < VSTS Feature Milestone 2005-07 "download/remote access only – First build with new thing X – only get if you want to see X"

    < VSTS August CTP "So now we did some testing too, go get it"

    < VSTS Beta 3

    < VSTS RC 1

    < VSTS RTM?

    Ok so not everyone may agree with that but I think that would be less vague than having monthly CTPs with different levels of quality. People have now come to expect CTPs atleast work if you’re in luck.. I’d like more frequent builds but they must be named properly to set the right expectations!

  6. As I mentioned before , we’re in the process of updating our internal version of VSTS with updated bits.

  7. 話題の小向美奈子ストリップを隠し撮り!入念なボディチェックをすり抜けて超小型カメラで撮影した神動画がアップ中!期間限定配信の衝撃的映像を見逃すな