Followup to post re: community preview philosphy

Thanks for the comments/feedback I received.  A few additional comments (I’ll try to keep this one short, or at least shorter).

Josh Ledgard  questioned the need for every/daily builds.  I use the word “every” to make a couple points, one internally and one externally. 

The internal point is that “shipping” these builds – with all necessary extra info like content and quality – has to be automatic/free.  If it’s not, i.e. we have to decide which build, or do extra work to “polish” the build before it ships, there will be resistance to doing them – the daily build is pretty ingrained into how we do things and keeping forks (and dev machines set up) for previous builds adds lots of work.  We also get into gnarly conversations about whether to fix some hideous bug in one area before we ship it – those are gnarly conversations because up until we get close to shipping, there’s almost always at least one hideous, embarrassing bug in every build – the choice is really to lockdown the build while we fix the hideous bugs, or fix it in the next build and risk that the new crop of hideous bugs is worse.  The longer we stay locked down, the longer most devs go without having their work subjected to the next testing gauntlet – they can’t check in, can’t build against all the other changes, etc..  Someone commented that Whidbey is complex – that is absolutely reflected here at the daily, product development level.

Externally, I want to make sure that we set expectations that the build from today is not necessarily better than the build from yesterday and that the casual observer should think twice before installing.  Our builds do not have a consistent incremental increase in quality.  There are spikes and valleys.  There are valleys even late in the release cycle, e.g. we send a beta out, get feedback, check in improvements across the product, and break things (which we then fix).  I want customers to see lots of builds coming out so they understand the risk, vs. seeing fewer builds and thinking (erroneously) that each build is good and “guaranteed” to be better than the last one. 

All that said, I’m more committed to frequent, automatic, and public, than “every build”.  I just think aiming for every build is the best way to get there – otherwise we’ll think incrementally.  It’s easy to scale back, e.g. from every day to once a week, or to whatever we end up seeing customers do. 

I do not mean to imply that there is zero order within our teams.  Over time the builds absolutely get better, we have a set of build verification suites that are run for every build,  we have tools in place to help make “breaking changes” easier to manage across the division, and there are always people looking at process/efficiency improvements.  In many ways, what our teams do is amazing.  But the complexity is very high, the % of tests we can run against each build is relatively small, and bad bugs get through.

The feedback about “must always pass”  tests is good.  That is what our build verification tests are meant to show, but reading this prompted me to think about three things: 1) maybe we’re not looking at the right things in the bvt’s.  2) maybe we need a more comprehensive set that takes longer to run but which still completes within a day or so.  3) can we get community help here?  i.e. perhaps we augment what we’re doing with a suite that the community provides.  That would be a great rallying data point for the division – do our daily builds at least reach the bar for what customers are willing to install?  I’m not looking to pass work off on others, just thinking about where I can blur the line between internal and external so that everyone ends up better off in the end.  I don’t remember if I’ve written about this before, but I’m also interested in the possibility of developing many test cases in public workspaces so people can at least see them.  Maybe bvts are the pilot we’ve been seeking.

Thanks for the comments re: encouraging people to try this.  That’s exactly where we are (although we’re also lucky to have strong support from Eric Rudder on down to help the division get started and survive through the inevitable glitches).  People are pretty excited about being more aligned with customers’ needs and wants.   I haven’t met a person who thinks this is a bad thing.  The concerns I have heard are very legitimate:  build quality is variable, how do we do this in a way that doesn’t cause customers to waste their time or  become dissatisfied, it’s extra work to gather/publish some of this information.  Those are all things to work through, vs. reasons for not doing it.     



Comments (11)

  1. I think the only obstacle to "every" build from the consumer perspective is size. If you can deliver an incremental "every" build in, say, under 50Mb, I would say go ahead and post every build. However, if the builds are going to be 2Gb+, then there’s just no way many people would go through the trouble of downloading, burning, and installing the monstrosity on a daily basis. (heck, even on a weekly basis!)

  2. Mark Cliggett says:

    We’re not trying to achieve a scenario where any one customer installs on a daily basis. It’s more about publishing them daily/weekly, and enabling customers to install when it makes sense to them. My hope – and I agree that size will impact this very significantly -is that most builds get enough installs to generate a customer-based assessment of "was this a good build". That way another customer who wants to install a recent build can look back and pick the best one based on the information available.

    I said this in the other post, but the scenario for me is a customer reports a bad/blocking bug to us, we fix it, and the customer can install the fixed build within a few days.

  3. AT says:

    Customers (read – me ;o) already proposed to increase pool of automated test cases.


    Visual Studio targeted to experienced users – not a home users. This will possibly allow you to ask them to provide automated test cases as part of repro steps.

    It’s pretty easy to solve security problem that Rob raised.

    You can run tests from each user in sandbox. You have powerful process security in NT, it’s no longer a Windows 9x world.

    Even more – if you have problems to deliver product to user PC – simply give them access to your servers. You have spend a lot of resources for pure marketing by giving away 180-minutes access to Visual Studio Terminal services (

    Use the same resources to expand VCCoreTS group. You will be able to solve problem of single build available to user – as you can keep several installed builds on different servers. Combined with access to automation tools this can result in something new and powerful.

    Okey. It’s one side of problem.

    Another part is lack of information that is going with product inside Microsoft.

    The only one source of new features/fixes is always outdated "What’s New" MSDN page and KnownIssues.html.

    Releasing new builds every week without updating those files will definitely result user not installing new build.

    ChangeLog is a must. BVT status is some kind of automated changelog – as changes can be both positive and negative

    <some fun> BTW, I’ve never seen in any ChangeLog file line like "Today we re-introduced 123 old bugs and 10 new one" ;o) BVT status will reveal this to users.

    </some fun>

    Also there is only a few future directions exposed and no time estimates given for bug fixes. Users unable to plan their time – this result in decreased user activity.

    More information on product progress is a must.

    You need to share information with customers – not CDs.

    P.S> As for making money aspect – you can benefit from Software Assurance licensing by allowing users with long term licenses to use select betas in production (if customer will agree to take all responsibility for this). Somewhat risky – but can be reviewed as an option to get more real-world scenarios during product development.

  4. Amr Essam says:

    I develop with both C# and VB.NET, but I like VB.NET

    to be better because I am old VB developer.

    I just installed VS.NET 2005 (Technology Preview), and

    this my prompt feedback


    I surprised that I found productive feature in C# and

    not in VB.NET, this features:

    1) Refactoring

    I tried Refactoring in C#, It is really very



    2) IntelliSense/Auto Completion (Keywords)

    Now only in C# debugger feel the Language keywords


    (private, public, foreach …), and auto complete it



    However VB.NET have longer Keywords such as




    I dreamt that I can find these features in VS.NET, but

    unfortunately I frustrated when I found this feature

    in C# not VB.NET 🙁

    Microsoft always say that it concentrate on

    productivity features in VB.NET, How does VB.NET 2005

    lack these very productive features ?!

    Looking forward for your reply.

    Amr Essam

    Consultant & Team Lead


    Dallas, Texas

  5. Joku says:

    Could it be cause VB.NET team focused on E&C, which is missing from C# as C# team focused on the refactoring and other little nice things.

    VB.NET has long keywords probably cause that’s how VB seems more approachable as it resembles something you know (english).

    What I don’t understand is that why there is no single "codebase" for VB.NET & C#, and then the language syntax differences between them would be just glued on using inheritance or whatever. Then when E&C was introduced into the "root code", both VB.NET and C# would have "inherited" it. I am probably over simplifying things in my mind..

  6. Eric Parker says:

    As a member of the general public and not special in any way, I’d just like to say that it is very difficult to hear about all the great new features of the product formally named Whidbey for over a year now, and still not be able to play with it at all! I feel like I am already left standing at the starting gate while the rest of the runners have run 17 laps before the gun has sounded! I have a 6 Mb connection to the internet and VMWare with XP Pro installed. It would be very, VERY cool if there was someway to download the preview, even though I am not an MSDN Universal subscriber, or MVP or whatever. That doesn’t necessarily mean I’m less interested or technically savvy! I mean, I’m practically dying from all the taunts. I’d really rather just not hear anything about it if I can’t get access to it for two years!

    Dry Mouth in Dallas.

  7. As a keen .Net developer without an MSDN Universal Subscription ts very frustrating not being able to get my hands on the VS2005 releases – are they going to make it out for general download?

  8. Stuart says:

    Just to reiterate what Martin Dolphin said – it’s great that you’re working towards making more of your builds available, but it would be even better if *something* were available to a larger portion of the general public. Not necessarily these daily builds, but something. Last I heard, the public beta was scheduled for "June" – June’s nearly over and there’s no new word, that I’ve heard. Any news on that?

    It’s very frustrating – I’ve been positively drooling for C# with generics, and I can’t get at it anywhere! Preview access to VS2005 excludes me, and the Mono windows installer doesn’t include their generic-enabled preview compiler (I’m not a sufficient guru to build Mono from source, especially on Windows).

  9. Also advance cash loan payday wired advance cash day loan pay

  10. In need of advance cash day loan pay cash advance service