Release Management planning update – 2016 H2
Now that we are done with a majority of items listed in our 2016 H1 plan, it is time to talk about our plans for 2016 H2. Most of the features listed below will first be available in VS Team Services, and they will be available on-premises in TFS vNext. We are quite early in these plans, and some of the screenshots shown below are just conceptual.
Let us start with a few remaining items that were already stated in 2016 H1 plans. These will be delivered in the next couple of months.
- REST APIs for Release Management
- Service hooks for Release Management
- Documentation for various UI contribution points in Release hub
A big focus for 2016 H2 is on simplifying application deployment. Here are some of the improvements that we plan to work on:
Agent-based deployment to machine groups
Our approach in Release management so far has been to integrate with various deployment tools and platforms while providing rich control over the flow of bits, traceability, and auditability. We have first-class integration with Azure, and for a majority of other deployments, we have provided the ability for you to run Powershell scripts either on the agent or on the target servers (using Powershell on target servers task). However, we have an overwhelming amount of feedback that we should do more in this space:
- Deploying releases on multiple machines (on-premises or Cloud IaaS) is non-trivial in the current approach as it requires a huge number of pre-requisites to be enabled – WinRM configuration, certificates, firewall ports, etc. The large number of issues that are reported to us in this area clearly indicates that there is more we can do in this space.
- Deploying releases on multiple machines in the current approach is not scalable especially if you need to deploy to a large number of machines. In particular, customers would want to perform rolling and canary deployments when deploying to a number of servers.
- Deploying releases to Linux servers is not easy.
- There is no traceability of releases all the way down to machines. In the current approach, success or failure of a deployment to a particular machine is buried in the logs.
Over the rest of the year, we plan to deprecate the current concept of machine groups as it exists in the Test Hub, and introduce a more robust multi-machine deployment capability that relies on the same build and deployment agents. Unlike the current approach, where you install the build and deployment agents on a set of proxy servers in an agent pool and drive deployments to remote target servers (or machine groups in Test hub), we will enhance our orchestration so that you can install the agent on each of your target servers directly, and then drive rolling deployments to those servers. You can use the same proven cross-platform agent and its pull-based execution model to easily drive deployments on a large number of servers no matter which domain they are on, without having to worry about the myriad of pre-requisites.
Here are a couple of screenshots (likely to change) to explain how we are thinking about this experience.
1. Install the ‘build and deployment agent’ on all machines in ‘Production’ machine group, and monitor them under the Machine groups tab in Release hub.
2. Deploy to machines in the machine group.
As part of this work, you will notice that we will gradually deprecate the current machine groups as they exist in Test hub. More specifically, we will:
- Ensure that all the remote Powershell and remote testing tasks work directly with a list of machines instead of current machine groups in Test hub.
- Introduce warnings that you should shift your build and release definitions to stop using current machine groups.
- Continue to support remote Powershell and remote testing tasks to work with a list of machines even after this change and after the introduction of the new more robust machine groups.
Simplifying Azure deployments
At present, some of the tasks (e.g., Azure webapp deployment) work only with Azure Classic endpoints, while other tasks work with Azure Resource Manager endpoints. Furthermore, there are a number of challenges with deploying to Azure:
- Setting up a connection to Azure Resource Manager requires running a Powershell script to generate a Service Principal.
- Azure webapp deployment uses Azure commandlets and, as a result, we are unable to effectively address some of the issues such as files being locked or deploying multiple virtual apps.
- Changing web app configurations in each environment currently requires a Marketplace extension to be added.
- Deploying releases through Azure webapp task does not publish deployment history to Azure.
- There is no flow that starts from Azure portal to setup continuous deployment via Build and Release management in VS Team Services.
We plan to start addressing these limitations by first simplifying how you can configure a connection to Azure from VS Team Services. Based on the subscriptions that are accessible to the logged in user, we will provide an seamless experience to create a new Azure Resource Manager endpoint in VS Team Services. We plan to re-implement the Azure webapp deployment task to be based on Azure Resource Manager endpoint and using webdeploy. We also plan to integrate tokenization and configuration transformations into that task. Finally, we plan to add a continuous deployment flow from Azure portal to use Build and Release management.
Docker and Windows container deployments
Docker and Windows containers are emerging as the new way of deploying cross platform applications through immutable images. We will start supporting these technologies through one or more extensions in the Visual Studio Marketplace. Through our extensions, you can build a new image, publish it to a Docker registry, run it on Linux or Windows hosts, and even use technologies such as docker-compose to compose multiple containers into an application.
In addition to application deployment, we will continue to enhance the release orchestration capabilities as follows.
Manual intervention steps
It is often the case that your deployment and testing is not entirely automated. For instance, you may still be deploying your database changes manually or testing manually. To cater to these requirements, we will allow you to introduce a manual intervention step in the middle of your environment workflow. We will automatically pause the workflow and release the agent at that point, and resume the workflow after the manual step is marked as complete.
Meta-tasks, folders, favorites, etc.
As more and more teams adopt Release management, and as their usage increases, we have been getting requests to make it simpler to manage multiple release definitions. If you are deploying several tens of applications using a similar release process, then, at present, you have to replicate the same automation process in many environments. To make this easier to manage, we are introducing meta-tasks – reusable task sequences – that you can add to any environment ‘by reference’. Editing a meta-task’s sequence or configuration in a central place would cause the change to happen in all environments, where it is used.
We will allow you to better organize your release definitions through the use of folders and tags. You should also be able to mark certain definitions as your favorites and view them more easily on the Release hub.
Audit history for releases
Another common ask, particularly from teams that are looking for meeting compliance requirements, has been to track all changes and deployments of a release. We plan to show all updates on a release to help meet this need.
Package management integration
Release management can be used presently to deploy artifacts from Team Build or Jenkins. Continuing down the path of being more open, we plan to publish an extensibility model for adding your own build and drop sources into Release management. As an example, we will showcase how you can integrate Release management with Team City.
Besides integrating with build systems, Release management can be used to directly deploy code from code repositories. For instance, if you are developing a PHP app, you should be able to use Release management to deploy code from the Git repo or TFVC repo where the source code for the PHP application is managed. Integrations with Git, Github, and TFVC sources are planned.
Package repository is another place to publish and manage versions of your artifacts. We plan to support consuming feeds from package management service as part of Release management. All of these integrations will help you track the versions of packages/drops/commits that are part of a given release.
The above are some examples of features that we have planned for you in the second half of this year. We have not list everything that we planned. But, if there are features that are important to you and are missing in the above list, call them out in the comments below.