Visual Studio Team Services Integration with Jenkins
Visual Studio Team Services (or Team Foundation Server) is a bundled suite of DevOps tools that can also integrate with other tools used by your team. Its REST APIs, hooks, and extension points are frequently leveraged to build custom integrations. Team Services also includes a growing list of preinstalled integrations. One of its integrations is with Jenkins, a leading automation server used for Continuous Integration (CI). Team Services integration with Jenkins allows using both CI systems with traceability through a single DevOps platform. This post summarizes Team Services’ initial capabilities of integrating with Jenkins. Supported scenarios include:
- Using Jenkins for continuous integration of Team Services Git repositories
- Using Jenkins to validate Team Services pull requests
- Mixing Jenkins and Team Services to perform builds and releases
- Providing traceability between Jenkins and Team Services; linking builds, pull requests, commits, and stories
Using Jenkins for Continuous Integration with Team Services
There are multiple ways to use Jenkins as a CI server with Team Services. Jenkins’ built-in Git Plugin or Team Foundation Server Plugin can poll a Team Services repository every few minutes and queue a job when changes are detected. For those who need tighter integration, Team Services provides two additional ways to achieve it: 1) the Jenkins Service Hook, and 2) Jenkins build and release tasks.
Jenkins Service Hook
The Jenkins Service Hook (Figure 1) in Team Services automatically queues a Jenkins job when code changes are pushed to your repository. This is an improvement over polling because jobs are immediately queued. For Team Services to queue a Jenkins job from the cloud, it requires access to the Jenkins server. If the Jenkins server is located behind a firewall, it must be exposed. This technique also makes it more difficult to trace the steps between a Jenkins build and Team Services artifacts. For example, a hyperlink must be followed from the Jenkins build to the Team Services commit to find user stories associated with the build.
Jenkins Build and Release Tasks
Team Services adds capabilities over the Jenkins Service Hook by including connectors that allow its build and release systems to integrate with Jenkins. These connectors can be chosen from the list of tasks to execute as steps in a Team Services build or release definition (Figure 2).
In effect, a Team Services build or release will queue a Jenkins job and download resulting artifacts. Because these tasks execute in a light-weight, long-polling agent that can be installed in your data center, there is no need to modify inbound firewall rules for Team Services to access your Jenkins server from the cloud.
The Jenkins Queue Job task (Figure 3) queues a Jenkins job by name. It can wait for the job to complete or immediately continue to subsequent steps. Multi-branch pipeline jobs and parameterized jobs are also supported.
When the option is selected to capture the Jenkins job’s console output, it is displayed live in Team Services while the build executes (Figure 4). After the build completes, the console output is available in the Team Services logs.
Downstream jobs and multi-branch pipeline results are displayed in the Team Services Build or Release Summary for traceability (Figure 5).
If your Jenkins server has the Team Foundation Server Plugin installed, a Jenkins post-build action named Collect results for TFS/Team Services makes test and code coverage results available to Team Services (Figure 6). Team Services’ Publish Test Results and Publish Code Coverage Results tasks (Figure 7) will publish these results to appear in Team Services build and test reports (Figure 8).
Jenkins credentials are configured with a Jenkins Service Endpoint that is shared throughout your team project (Figure 9).
The Jenkins Download Artifacts task (Figure 10) makes it easy to download build artifacts from Jenkins and integrate them into your Team Services build or release process. Once artifacts are downloaded, they can be used by other tasks as demonstrated in the video below.
Using Jenkins to Validate Pull Requests in Team Services
Team Services allows creating branch policies that require a successful build before any code can be submitted into a branch (Figure 11). For Git repositories, this requires using a pull request to merge changes into the configured branch.
A Jenkins job can be used to validate pull requests by setting the branch policy to use a Team Services build that queues the Jenkins job using the Jenkins Queue Job build task. In this scenario, Jenkins will build the merge commit of the pull request. Completion of the pull request can be gated upon the successful outcome of the Jenkins build (Figure 12).
This scenario requires configuring the Git Refspec field in Jenkins to build the merge commits created for pull requests in Team Services (Figure 13; further described on GitHub).
Mixing Jenkins and Team Services Builds and Releases
The previous examples describe using Team Services to queue a Jenkins job and download resulting artifacts for subsequent use within a Team Services build or release definition. Jenkins can also independently trigger a Team Services release. After installing the Team Foundation Server Plugin on your Jenkins server, you can configure the Jenkins post-build action named Trigger release in TFS/Team Services to initiate the release (figure 14).
Next, set your Team Services release definition to use artifacts from the Jenkins build (Figure 15). Artifacts will be downloaded from Jenkins and used in your Team Services release pipeline.
Traceability Between Jenkins and Team Services
When integrating Team Services with Jenkins, traceability is important for knowing which Jenkins builds are associated with Team Services artifacts. Bidirectional linkage is provided between Jenkins builds and Team Services repositories, builds, releases, pull requests, commits/changesets, stories, and issues.
You can maximize traceability by installing the Team Foundation Server Plugin on your Jenkins server and configuring two post-build actions (Figure 16).
Traceability is available as follows:
Links from Team Services to Jenkins
The Team Services repository status shows whether the last Jenkins build succeeded for the current branch and links to recent Jenkins builds for the branch:
Team Services build and release summaries link to their corresponding Jenkins builds:
Team Services pull requests link to the Jenkins build that validated them:
Team Services work items (such as user stories and issues) link to the Jenkins builds in which their associated commits and pull requests were built:
Links from Jenkins to Team Services
A Jenkins build links to the Team Services build that queued it:
A Jenkins build links to the Team Services branches and commits that were built:
A Jenkins build links to its associated Team Services pull request and work items (such as user stories and issues):
How Can We Improve?
We welcome your suggestions for improving Team Services integration with Jenkins. We are continuing to enhance it and already have a few ideas:
- A Jenkins action that queues a Team Services build
- Jenkins Service Hook support for validating pull requests with a Jenkins build
- Automatic publishing of test and code coverage results to Team Services when produced by a Jenkins build
The Jenkins Queue Job and Jenkins Download Artifacts tasks are open source. Contributions are welcome, and you can use these patterns to create your own extensions in the Team Services Marketplace. To make suggestions or report issues, reach out to us on GitHub.
For information about Team Services integrations for Java, including plugins for Eclipse, IntelliJ IDEA, and Android Studio, visit java.visualstudio.com.