Moving TFS to cloud cadence and Visual Studio Online


We get quite a few questions from customers on how we made the transition to shipping both an on-premises product and a cloud service. We moved from shipping every 2-3 years to shipping Visual Studio Online every three weeks and TFS every 3-4 months. You’ve probably seen the great set of vignettes at Scaling Agile across the Enterprise. It was also recently covered in a report from McKinsey and in an interview on Forbes.com. What’s missing is a deeper description of what changes we’ve made to how we work.

A couple of years ago, we wrote document on how the team changed to meet the new demands of building a service. It will give you a lot more information on what we did and how for anyone who wants to go deeper. I’ve cleaned up the document by converting our internal terminology, but it’s essentially unchanged from when it was written. Here is the summary from the document, and you will find the entire document attached to this blog post as a PDF.

The adoption of Scrum for TFS 2012 was driven by our desire to deliver experiences incrementally, incorporate customer feedback on completed work before starting new experiences, and to work like our customers in order to build a great experience for teams using Scrum. We used team training, wikis, and a couple of pilot teams to start the adoption process.

We organized our work in four pillars of cloud, raving fans, agile, and feedback with each having a prioritized backlog of experiences. Teams progress through the backlog in priority order, working on a small number of experiences at any point in time. When starting an experience, teams break down the experience into user stories and meet with leadership for an experience review. Each three-week sprint starts with a kick off email from each team, describing what will be built. At the end of the sprint, each team sends a completion email describing what was completed and produces a demo video of what they built. We hold feature team chats after every other sprint to understand each team’s challenges and plans, identify gaps, and ensure a time for an interactive discussion. On a larger scale, we do ALM pillar reviews to ensure end to end cohesive scenarios.

With the first deployment of tfspreview.com in April 2011, we began our journey to cloud cadence. After starting with major and minor releases, we quickly realized that shipping frequently would reduce risk and increase agility. Our high-level planning for the service follows an 18 month road map and a six month fall/spring release plan in alignment with Azure. To control disclosure, we use feature flags to control which customers can access new features.

Our engineering process emphasizes frequent checkins with as much of the team as possible working in one branch and using feature branches for disruptive work. We optimize for high quality checkins with a gated checkin build-only system and a rolling self-test system that includes upgrade tests. During verification week, we deploy the sprint update to a “golden instance” that is similar to production. Finally, we ensure continuous investment in engineering initiatives through engineering backlog.

We’ve made a number of changes in the last couple of years, and I’ll write about those in upcoming posts. The biggest changes have been in our engineering system and in our organizational structure. We now have the TFS/VSO team in Visual Studio Online. When we moved into VSO, we also moved from TF version control into Git. As we previously moved to Scrum in part to be able to build and use experiences needed for Scrum teams (and now Kanban as well), we wanted to ensure we build a great experience for Git and that we also derive benefits from the workflows that it enables, including being able to easily branch, do work, and commit the change back into master. TFS 2015 CTP1 and Team Explorer in VS 2015 CTP6 are the first releases we’ve made to the on-premises products from this new engineering system.

We’ve also changed the organizational structure by combining development and testing into a single engineering discipline. We made the change four months ago, and we are still learning.

Where we are now is also not where we want to be. For example, our architecture clearly shows our on-premises software origins, and we still have work to do including splitting out version control, build, work item tracking, and test case management into separate, independent services. At the same time, we need to collapse them back into a product that’s easy to run on-premises (service hooks, for example, is a separate service in VSO that is going to ship in TFS 2015). It’s an evolution – making changes to the product while continuing to ship valuable features for both the cloud and on-premises customers, all from a common code base owned by a single team (rather than separate cloud and on-premises teams).

I’m sure this post will create many questions, which will build a backlog of posts for me.

[Update March 12, 2015] Brian has written a new post about the future of Team Foundation Version Control.

Follow me on Twitter at twitter.com/tfsbuck

Moving TFS to Cloud Cadence.pdf

Comments (11)

  1. Philippe says:

    >we also moved from TF version control into Git.

    I am highly interested by the way you did your migration! Did you migrate all the history? Which tool did you use? Will Microsoft, one day, provide a tool to convert a TFVC project to a git one?

    We want to convert our project but at the moment, it's such a pain….

  2. Carl D says:

    Nice post, Buck!

  3. Kim says:

    That explains why all the new stuff goes into GIT control. How do you make sure that we, who uses TFS source control, also get some attention and new stuff?

    The problem with this high cadence for the on-presmise, is that it is hard to stay current if you are using the full monthy, like lab and test and the relative little supportmatrix of all products(TFS, hyper-v, sql, SCVMM, SharePoint and OS) when combined.

  4. Roger G says:

    Where is Team Project rename? Is it being delivered in 1 sprint or across sprints?

    Where are updates to MTM and CodedUI?

    How do you, (do you) use TFS Lab Management for testing with the new discipline "combining development and testing into a single engineering discipline."?

  5. Buck Hodges says:

    Philippe, we did not move history. We actually took the latest code from TFVC and checked it into Git. Now, the transition occurred over multiple weeks, so we did this repeatedly until everyone was in Git. We could have used tf-git, though I don't know that it would have scaled to the size of our code base. We decided to leave history behind in part so that we could have smaller repos. In our initial move, we split the code base into a couple of dozen repos. We collapsed it into one repo back in the fall, and that's worked much better for us now. I need to write a post on this. One other thing of note is that we are working on supporting both TFVC and Git in the same team project.

  6. Buck Hodges says:

    Kim, you see a lot of new stuff going into Git because our Git support is not at parity with TFVC. TFVC has over ten years of work in it, while we still need key Git functionality from rebase in the VS GUI to more integrated Git pull request workflows.

    I helped build TFVC as a developer (it is by far the largest topic on my blog over the last 11 years). TFVC is an amazingly scalable centralized version control system. We are adding new features to it. One of the things that we are working on for TFS 2015 is a revised code review experience, and that will bring new functionality into TFVC.

    There's an interesting blog post that was written recently that's interesting reading on DVCS vs. centrailized and the tradeoffs: bitquabit.com/…/unorthodocs-abandon-your-dvcs-and-return-to-sanity Both DVCS and centralized have their advantages.

    I agree that when you add up the products across multiple teams that with all the different requirements that the common set is relatively narrow. For updates to a given TFS major release, we don't change the system requirements for that reason. I also believe (assuming I'm not forgetting something) that the system requirements for TFS 2015 should be almost identical TFS 2013 (unlike some prior releases where we changed quite a bit).

  7. Buck Hodges says:

    Roger, team project rename is going to be in TFS 2015 RTM. It was not ready for the CTP we just released, which is why you don't see it there. It is definitely a multi-sprint effort (six or so).

    I don't have news for you on MTM and CodedUI features right now (I'll ask that team).

    We don't currently use TFS Lab Management to do our testing. We use an internal test VM cloud. I hate to say that because one of my own goals is that we don't use internal tools for our own ALM needs (internal tools don't help our customers!), but the test space is one where we use some and don't use others. We've had a good amount of conversation about it recently, and there are some pilot projects to start using more of these tools, which will be good.

  8. gwern says:

    Really surprised you guys are using your own internal test VM cloud and not lab management. You are right that your internal tools don't help customers and lab management/Microsoft's testing tools (codedui / mtm) have really really atrophied over the last 3 years from lack of investment and just pure neglect…sure you have improved the manual test case authoring experience to a degree…grid view in the web…but I'd be great to learn more about the tooling you are in fact using day to day for your integrated testing. Again if it's internal…it's not helping customers… and as a customer who's been on the TFS train since 2010, we are now having to look at other vendors and tool chains like UFT from HP….which decreases the "value" and our reliance on our investment in TFS. Making the case to stay on TFS and now switch…is hard….and your competitors are bringing compelling, all-be-it expensive, offerings to market in the integrated test space. Thanks for keeping the comments and dialog open.

  9. Shamik Basu MS says:

    Gwern, the test feature team is building a significant set of functionality for the VS 2015 release, some of which is released in updates already and in VSO, the rest of it is coming in subsequent CTPs. This will broaden the Testing functionality we have. E.g. regarding the distributed lab management scenario you mention, in the Automated Testing scenario we will provide the ability to run large # of diverse tests (Unit, Functional (CUIT/Selenium), Integration, …) on multiple machines, seamless integration with Build and Release Management, simpler lab management experience in TWA, support for Azure environments and hybrid topologies, and ability to run 3rd party adapter tests and publish results to TFS. This functionality did not ship in CTP1 but will be included in subsequent CTP releases and you will get to try out these features sooner on VSO. We will also be adding authoring, execution and debugging functionality for unit tests and Coded UI for Windows 10 Universal apps on PC’s, phones and emulators.

    We have expanded our web-based Test case management functionality on several fronts to allow testers to access this functionality from everywhere. As part of CTP1 we have expanded the Basic license to include web-based test execution and assign multiple testers to a test suite and invite them for testing, this is described at http://www.visualstudio.com/…/tfs2015-vs. The following features from 2013 updates have also been rolled up – test case Web view improvements like column options, exporting test artifacts, shared parameters for test cases, test plan and test suite customization, allowing bugs on the backlog, associated test suites, recent test results and test case charting to allow you to visualize test planning and test results. Other testing features over the past few updates include Exploratory Test support, enhancing manual test functionality, test plan/suite cloning etc. all documented at blogs.msdn.com/…/visual-studio-and-tfs-2013-4-update-4-release-candidate.aspx and blogs.msdn.com/…/the-effect-of-updates.aspx.

    Load testing is another area we have focused on a lot given the increased cadence of releases with DevOps and how this has become a much more integral part of regular test processes. We have added geo-targeting to our cloud load service, built up the service to handle significantly higher-scale loads – and are in the midst of upping this even further.

    We are very committed to providing world-class testing functionality, please do let us know any pain points and feature asks you have.

  10. Philippe says:

    >We decided to leave history behind in part so that we could have smaller repos.

    Not every team could be able to leave there history behind :-( Because history is one of the most important thing of a project and because you could have running branches when you try to migrate…

    As one of the two main developers of the tool 'git-tfs', let me know if you want to help us make a descent tool to be able to migrate the history from TFVC to git…

    > In our initial move, we split the code base into a couple of dozen repos. We collapsed it into one repo back in the fall, and that's worked much better for us now. I need to write a post on this.

    Indeed ;-)

  11. gwern says:

    @Shamik, thank you for your response. In reading it I have to say it comes across as being completely disingenuous and very tone deaf….if you have a blog that's going to go deep on the investments and prioritization of features coming in the 2015 release…I'd like to engage there..

    I have so so many questions about the viability of Microsoft's testing technologies and platform.

    1) What list of actual improvements, and or features, can you list, that have been or will be delivered to the MTM client itself? <From 2013 to 2015>

    2) What list of actual improvements, and or features, can you list, to codedUI (being able to author automation scripts in MTM, merging UI Maps, handling object recognition by image, running cross platform on mac, android, and iOS)?

    2b) You mention < debugging functionality for unit tests and Coded UI for Windows 10 Universal apps on PC’s, phones and emulators> what about cross platform codedui for Android and iOS…Window 10…hell Universal Apps aren't available yet…and won't be till later this year…but Android and iOS are here NOW…and we NEED automated testing tool that work cross platform.

    3) What 3rd party vendors have delivered products that extend with MTM (there hasn't been a public SDK will there be one for 2015)?

    4) With regard to "distributed lab management"will 2015 deliver support for running on Amazon EC2 VMWare/Xen virtual environments? Will distributed lab management environments be able to be a mix of on-prem and cloud in the same environment?

    5) What list of actual improvements, and or features, can you list, for exploratory testing? You delivered exploratory testing in VS 2012, and it hasn't been improved since, for example will it be possibly to run exploratory tests from web TCM in 2015?

    6) With regard to load testing, have you delivered real world, mobile device load testing features?

    7)What list of actual improvements, and or features, can you list, for MSTest as a testing framework? Every indication is that nunit is light years ahead of MSTest…how will MSTest reach parity with nunit?

    8) Are you guys reading your own uservoice site? There are so many pain points and feature requests that have gone un-noticed. visualstudio.uservoice.com/…/30927-visual-studio-test-and-lab-management

    If you are "very committed" to providing world-class testing functionality then deliver a compelling 2015 upgrade to MTM.

Skip to main content