Goals for Community Tech Previews


Mark, my boss, posted a long article that explains the intent and direction of the CTP program.  Here are some choice quotes and some of my own comments…


Our ideal is that every build is publicly available.  Another way to say it is that we want to enable a scenario where a customer finds a bug, reports it, the team/developer gets the report essentially immediately, sees it’s an easy fix, fixes it, resolves the bug, the customer gets notified, and with no extra effort from anyone (beyond the customer installing) the customer can install the fixed version within a few days.


First, we need to solve the distribution problem.  We’re looking at several different options – shipping media, VPC images, the p2p solution suggested earlier.


Another solution is obviously looking to reduce the size of the download a few different ways.


Second, we have to expose what we know about the build’s quality.  Internally we survive by running build verification tests on every build.


Third, we need to be smart about letting the community help in communicating build quality.


Coming from the test world; what amuses me about these statements is that, if we pull it off well, the external page would actually become the preferred location for internal people trying to figure out what build to install.  Yes, we do run several very basic build verification tests and some more intense “nightly“ automation if the first round passes, but the internal problem is that a lot of this data currently stays within the separate teams that ran the tests.  I’d been pushing for a while that if we thought about having to make our tests and results publicly consumable that it would actually lead to some much needed improvements to how we think about and report our collective testing output.  Its sort of a lesson for anyone thinking about the complications of an internal process to pretend your customers had to make use of the process.  I think it will make us more successful.  It won’t just be externals leaving their comments on the builds there either. 🙂


Fourth, we need to be more transparent about where we are in the dev cycle, what features are coming in now, etc… In the coming months expect us to do things like cover the project status on Channel 9 and be even more explicit in blogs.


We’ve thought about posting bug tallies, feature completion status, specific bug communications, etc.  What type of Channel9 information would you like to see? 


Lastly, we need to give customers a dependable, effective mechanism for submitting feedback and finding out what we’ve done with it.



  • We will have shifted much of the control/decision-making over how involved customers can be from us to our customers. 
  • It will enable every person within our division to be much more responsive to customer feedback, without a lot of incremental extra effort.

I’ll add that, personally, I’d like to see us untie the tools from the framework so that you could use stable builds from Orcas with it’s new IDE features for working with the Whidbey framework.  This, IMO, would also have a customer benefit in addition to enabling more people to test each build.

Comments (5)

  1. Hm, now I understand. It seems like you consider customers as another test side or BVT side. And feedback on quality – is what you expect, as far as I understand from your post.

    I thought (tell me if I’m wrong) that the main goal of CP is different. I understand it that the main goal of customer feedback is influence the framework development and the way IDE looks like.

    After all you guys don’t write and ship real apps written with your tools on the daily basis day-to-day for 2 years and ship product after product. I know you make real app sessions and you can imagine the way it would be but it is just not the same. You can take .net 1.0 as example.

  2. josh ledgard says:

    Actually that’s a big worry of mine. That there isn’t much in it for the customers. This is why I suggested that what we really needed to do was Untie the Framework from the product so that they could be serviced more separately and customers could see more frequent, and stable, tool releases.

    In the eclipse world, for example, customers happily use the Milestone releases because they can with their current apps and they also get the advantages of seeing the issues fixed (in stable) releases more frequently.

    I went off on the test side because, the truth is, we don’t currently have a good way internally do present product quality (IMO) and I want that solved regardless of whether or not the customers see it. I was just thinking it would be interesting if the reason we solved it was to give customers the idea of which builds where best.

    I actually might question the usefulness of having a daily drop available. I think weekly with a stable drop every 2-3 months and point release on some longer stretch would be most useful.

    All that said, product feedback, whether it be on the quality, or the direction of IDE features is the goal. I don’t find them to be exclusive. For example: you may give us feedback today that changes what a feature looks like or how it acts. A while from now, if we release a CP just before a beta and a lot of customers tell us quality has actually declined we might have to reconsider where we are in the product cycle and whether or not a wide beta should be delayed to fix more issues. Both pieces of feedback were important.

  3. Would it also be possible to make getting bug fixes easier?

    For example, today I ran into a problem with SourceSafe where it can’t get old versions of files using the command line version (commands like "ss -v "$/dev/file"" fail with "Version not found” in some cases).

    It appears that it is a known issue and that it has been fixed:

    http://support.microsoft.com/?id=822845

    The only problem is I can’t get the fix. Instead of a handy download link, I would have to call Microsoft support which for various reasons is a lot of hassle (I might need a credit card number and I would probably have to find the person who does product licensing to give a licence number etc and it would probably take longer than clicking on a link).

    I can understand that these fixes might not be that well tested but surely a big disclaimer and perhaps a change to the version information in the patch programs to something like "6.0d(a) UNSUPPORTED VERSION" would be enough.

  4. josh ledgard says:

    Yup, this is also something I have personally expeirinced. We are working with PSS to see if we can make these changes to hotfixes for our products are just availible without calling. Your not alone in your complaint.