Lab Management Evolution
[August 2015]
Lab Management was introduced in TFS 2010 to help development teams easily deploy and test their applications on virtual machines in their routine ALM workflows. However, you have not seen us add significant new features to Lab Management in subsequent releases of TFS. Many of you asked us about the evolution of Lab Management. In this article, we will describe how we are thinking about this area, and what you can expect over the next few months.
When we started building Lab Management features for TFS 2010, our focus was on integrating with one platform – Windows Hyper-V managed through System Center Virtual Machine Manager (SCVMM). A significant portion of our investment at that time was focused on filling the gaps that existed in SCVMM. Here are some examples:
- There was no concept of multi-tenancy in SCVMM at that time. The notion of a private cloud was just emerging. To integrate with TFS, we had to fill this gap by inventing a number of concepts for mapping different SCVMM host groups and library servers to team project collections and team projects. Contrast that with the present day, where there are a number of mature private and public cloud platforms, all of which have support for multi-tenancy in the form of accounts, subscriptions, and RBAC. The older model of integrating TFS with these cloud platforms does not make sense any more.
- Very few resource types were supported by SCVMM at that time – virtual machines being the prominent one. In fact, there was not even a concept of a ‘group of virtual machines’ in SCVMM. It was not possible to clone virtual machines without causing network conflicts. To support deployment of multi-tier applications from TFS, we had to fill that gap by introducing the concept of a “SCVMM environment”. And, we invested quite a bit in isolating one environment from another so that cloning could be done. Contrast that with today’s situation, where all the mature cloud platforms including SCVMM support a variety of resource types and have ways of grouping resources. For instance, Azure supports websites, storage, virtual machines, resource groups, and so on. Virtual networking support in these platforms can accomplish what we tried to do with network isolation in lab management, and much more. It does not make sense for us to get into resource grouping and networking in TFS anymore.
- If you think of a standard lab manager persona, he or she would be interested in setting policies on resource consumption (e.g., no developer should get more than 3 medium sized VMs), and on tracking that consumption (e.g., number of unused VMs). We have not focused on such lab management scenarios in the past either. We expect platforms and platform partners to provide such functionality, as they cater more and more to the needs of Dev/Test users. In particular, the Azure Dev/Test Labs effort is one such example. In the foreseeable future, we do not see a need to invest in such areas within TFS/VSO.
It is not just the cloud platforms that have evolved. The push for agility and DevOps also resulted in a number of light-weight portable deployment technologies. Container technologies such as Docker and wrappers such as Vagrant help developers easily provision new machines (or containers) on demand. These technologies are easy to use, easy to script, and fast to update. Moreover, one can deploy the same container or wrapper to staging and production environments. For teams adopting these technologies, lab management needs – in terms of easily provisioning new machines on demand – are mostly met.
Now that we have established what we do not want to do in the future, let us take a look at some of the requirements that have only become stronger now:
- Variety of platforms: The variety of platforms on which applications are being developed has grown. You have been asking us to build ALM tools that can help you easily integrate with private cloud platforms such as System Center, Azure Stack, and VMware, or public cloud platforms such as Azure and AWS. These platforms support a variety of resource types, and have made great strides in terms of infrastructure programmability.
- Automated ALM workflows: With the increasing need for faster and reliable deployments, the need for seamlessly provisioning fresh resources, deploying new builds, and testing them on an automated basis has increased. Some of you want to do this every time a new build is available, others want to do this on a nightly basis, and the remaining on a more frequent cadence than they are at currently.
- Manual ALM workflows: The need for development teams to setup their self-service, bug-bash, demo, or feedback environments has only increased in the DevOps world. Infact, with the DevOps mindset, the manner in which newer builds are deployed to any of these environments should be identical to the way they are deployed eventually to staging and production environments.
Within VSO and TFS, our services around Build, Test, and Release Management have been evolving rapidly as well. You have given us feedback that it is confusing to have environments in both lab management and in release management, especially when they do not integrate. You have also told us that we should simplify automated and manual ALM workflows significantly and that we should enable your teams to grow up from one use case to another. Moving forward, we will expose lab management features as part of the web interface for TFS, instead of in Microsoft Test Manager (MTM). In the rest of this article, we will show you how you can accomplish the automated and manual ALM workflows in this new world. Most of the features that we describe below will be coming over the next few months to you.
Azure integration
You can dynamically provision websites, cloud services, or resource groups in Azure. To do this, you will be able to add your Azure subscription to your VSO account, and start using that subscription as part of tasks in Build and Release Management flows. The resources can be used to deploy your application-under-test or to spin up a test agent and run functional tests.
We will support multiple forms of authentication from your VSO account to Azure – Azure certificates, user credentials, and service principals. With Azure resource group task, you can leverage the full power of Azure’s infrastructure-as-code. Start with hundreds of starter templates that are being defined by the community for Azure, and customize them to build the infrastructure that you need for your apps. Or else, you can use the Azure powershell task to script the resource provisioning commands.
Service endpoint extensibility
Azure is just one of the service endpoints that you can connect to from VSO. We have been working on a model, where customers and partners can contribute extensions that define new endpoint types for a different platform or deployment service, and that can run provisioning or deployment tasks against that service. We have started building some of these common extensions ourselves – Chef is one such example.
Integration with various platform and deployment technologies
Using the extension model, we plan to support various integrations out-of-the-box. Two integrations that we would like to call out as being in our plans are VMware and Docker – VMware for being a consistent ask from you, and Docker for supporting modern DevOps practices.
Machine groups
If you do not find your favorite platform integrated into VSO as an extension, you can still use “standard” machine groups as a way of describing a group of addressable machines. These machine groups can then be used as part of various deployment and testing tasks in Build and Release workflows. Just think of machine groups as the “catch all platform” that is always available to you within VSO/TFS. This concept used to be called “Standard environments” in Lab Management.
.Net and Windows application deployment
We will enable you to deploy .Net applications (IIS, SQL server, Windows services) easily to machine groups defined in VSO/TFS, to pre-provisioned resource groups managed by Azure or other integrated platforms, or to dynamically provisioned resources on various platforms as part of the same automation. Support for DACPAC deployments, IIS web application configuration, or running remote Powershell scripts on the target servers will be provided. If you have already decided on third party deployment systems such as Octopus Deploy, we will have integrations with those as well through the extension model.
Java and Linux application deployment
We will enable you to deploy Tomcat applications easily to resources on various platforms. If you have written shell scripts to deploy your apps to Linux, you can use our xplat agent to run those scripts. You can also deploy your Linux applications using Maven scripts or Ant scripts. All of these tasks are readily available out-of-the-box in your Build and Release workflows. If you have already decided on deployment systems such as Chef or Docker, we will have integrations with those as well.
Environments
Using a release definition you can model a formal pipeline of test, staging, and production environments. Each of these environments in Release Management is a logical entity that describes how resources should be provisioned, how new builds should be deployed, and how tests should be run against the resources of that environment.
If you want less structured and self-service environments for your manual workflows, then we will soon enable you to create release definitions without a formal pipeline. Simply group all the related self-service environments of your team members into a release definition, and you are now setup to use the same steps to deploy the next build to your own environment as to the production environment.
Traceability
Want to know which release is deployed currently on your personal environment, or the status of testing of a release on various environments? You can easily track all the activity on environments or on releases.
Summary
To summarize, we are moving away from building platform-style features and focusing more on integrating those platforms into ALM workflows. Lab management is morphing into being an integral part of Build and Release Management – in the form of service endpoints, provisioning tasks, deployment tasks, and testing tasks. We are increasing our investment in creating new service endpoints and in adding provisioning, deployment, and testing tasks in order to help you easily consume platform features and to make it easy for you to add your own integrations.
FAQ
When can I expect to see these features in VSO and TFS?
Some of these will be in VSO later this year, and the rest during 2016. In particular, we are working to complete Azure integration, service endpoint extensibility, and Windows/.Net application deployments before the end of this year. You can expect to see the same features in TFS early next year. The extensibility will help us and our partners to deliver newer integrations throughout 2016.
I am a current Lab Management user in TFS 201*. What does this mean to me immediately and in the long run?
As you might have noticed, we are continuing to ship Lab Center as part of Microsoft Test Manager (MTM) 2015 and you can still run Build-Deploy-Test workflows in Visual Studio 2015 using XAML build templates. We will continue to ship these features as part of 2015 updates with fixes to any major issues that are reported by you. And, we will continue to support the older versions of these products as well as the 2015 version as per the EULA terms.
But, you can safely assume that we will not invest on new features in MTM Lab Center, nor in enhancing the XAML build templates. We will also not bring in any of the above features such as integrations with newer platforms, integration with newer versions of SCVMM, or integration with the new Build and Release Management services into MTM Lab Center. Similarly, we will not integrate the current Lab environments from MTM into the new Build and Release Management services. We will probably not ship another major version of MTM Lab Center.
Once we have integration with SCVMM (or Azure Stack) in the new Build and Release Management services, we will strongly recommend that you start using those features in the TFS or VSO web interface for your automated and manual ALM workflows as explained in this article. We will separately provide further migration guidance to help you re-create equivalent assets in the new services.
Light
Dark
0 comments