Visual Studio Team Services Integration with Jenkins

Introduction

Visual Studio Team Services (or Team Foundation Server) is a bundled suite of DevOps tools that can also integrate with other tools used by your team.  Its REST APIs, hooks, and extension points are frequently leveraged to build custom integrations.  Team Services also includes a growing list of preinstalled integrations.  One of its integrations is with Jenkins, a leading automation server used for Continuous Integration (CI).  Team Services integration with Jenkins allows using both CI systems with traceability through a single DevOps platform.  This post summarizes Team Services’ initial capabilities of integrating with Jenkins.  Supported scenarios include:

  • Using Jenkins for continuous integration of Team Services Git repositories
  • Using Jenkins to validate Team Services pull requests
  • Mixing Jenkins and Team Services to perform builds and releases
  • Providing traceability between Jenkins and Team Services; linking builds, pull requests, commits, and stories

Using Jenkins for Continuous Integration with Team Services

There are multiple ways to use Jenkins as a CI server with Team Services.  Jenkins’ built-in Git Plugin or Team Foundation Server Plugin can poll a Team Services repository every few minutes and queue a job when changes are detected.  For those who need tighter integration, Team Services provides two additional ways to achieve it: 1) the Jenkins Service Hook, and 2) Jenkins build and release tasks.

Jenkins Service Hook

The Jenkins Service Hook (Figure 1) in Team Services automatically queues a Jenkins job when code changes are pushed to your repository.  This is an improvement over polling because jobs are immediately queued.  For Team Services to queue a Jenkins job from the cloud, it requires access to the Jenkins server.  If the Jenkins server is located behind a firewall, it must be exposed. This technique also makes it more difficult to trace the steps between a Jenkins build and Team Services artifacts.  For example, a hyperlink must be followed from the Jenkins build to the Team Services commit to find user stories associated with the build.

Jenkins Service Hook subscription
Figure 1

Jenkins Build and Release Tasks

Team Services adds capabilities over the Jenkins Service Hook by including connectors that allow its build and release systems to integrate with Jenkins.  These connectors can be chosen from the list of tasks to execute as steps in a Team Services build or release definition (Figure 2).

Add Jenkins build tasks
Figure 2

In effect, a Team Services build or release will queue a Jenkins job and download resulting artifacts.  Because these tasks execute in a light-weight, long-polling agent that can be installed in your data center, there is no need to modify inbound firewall rules for Team Services to access your Jenkins server from the cloud.

The Jenkins Queue Job task (Figure 3) queues a Jenkins job by name.  It can wait for the job to complete or immediately continue to subsequent steps.  Multi-branch pipeline jobs and parameterized jobs are also supported.

Queue Jenkins Job task
Figure 3

When the option is selected to capture the Jenkins job’s console output, it is displayed live in Team Services while the build executes (Figure 4).  After the build completes, the console output is available in the Team Services logs.

Jenkins Queue Job Console Output
Figure 4

Downstream jobs and multi-branch pipeline results are displayed in the Team Services Build or Release Summary for traceability (Figure 5).

Jenkins multi-branch pipeline job results
Figure 5

If your Jenkins server has the Team Foundation Server Plugin installed, a Jenkins post-build action named Collect results for TFS/Team Services makes test and code coverage results available to Team Services (Figure 6).  Team Services’ Publish Test Results and Publish Code Coverage Results tasks (Figure 7) will publish these results to appear in Team Services build and test reports (Figure 8).

Jenkins post-build actions
Figure 6

Publish test results from Jenkins
Figure 7

Build summary test and code coverage results
Figure 8

Jenkins credentials are configured with a Jenkins Service Endpoint that is shared throughout your team project (Figure 9).

Jenkins service endpoint
Figure 9

The Jenkins Download Artifacts task (Figure 10) makes it easy to download build artifacts from Jenkins and integrate them into your Team Services build or release process.  Once artifacts are downloaded, they can be used by other tasks as demonstrated in the video below.

Jenkins Download Artifacts task
Figure 10

Using Jenkins to Validate Pull Requests in Team Services

Team Services allows creating branch policies that require a successful build before any code can be submitted into a branch (Figure 11).  For Git repositories, this requires using a pull request to merge changes into the configured branch.

Branch Policy for Jenkins Pull Request
Figure 11

A Jenkins job can be used to validate pull requests by setting the branch policy to use a Team Services build that queues the Jenkins job using the Jenkins Queue Job build task.  In this scenario, Jenkins will build the merge commit of the pull request.  Completion of the pull request can be gated upon the successful outcome of the Jenkins build (Figure 12).

Pull Request Build Succeeded
Figure 12

This scenario requires configuring the Git Refspec field in Jenkins to build the merge commits created for pull requests in Team Services (Figure 13; further described on GitHub).

Jenkins Git Refspec Setting
Figure 13

Mixing Jenkins and Team Services Builds and Releases

The previous examples describe using Team Services to queue a Jenkins job and download resulting artifacts for subsequent use within a Team Services build or release definition.  Jenkins can also independently trigger a Team Services release.  After installing the Team Foundation Server Plugin on your Jenkins server, you can configure the Jenkins post-build action named Trigger release in TFS/Team Services to initiate the release (figure 14).

Jenkins Trigger Release
Figure 14

Next, set your Team Services release definition to use artifacts from the Jenkins build (Figure 15).  Artifacts will be downloaded from Jenkins and used in your Team Services release pipeline.

Jenkins Artifact Source for Release Definition
Figure 15

Traceability Between Jenkins and Team Services

When integrating Team Services with Jenkins, traceability is important for knowing which Jenkins builds are associated with Team Services artifacts.  Bidirectional linkage is provided between Jenkins builds and Team Services repositories, builds, releases, pull requests, commits/changesets, stories, and issues.

You can maximize traceability by installing the Team Foundation Server Plugin on your Jenkins server and configuring two post-build actions (Figure 16).

Jenkins Traceability Post-build Actions
Figure 16

Traceability is available as follows:

Links from Team Services to Jenkins

The Team Services repository status shows whether the last Jenkins build succeeded for the current branch and links to recent Jenkins builds for the branch:

Jenkins Branch Build Status

Team Services build and release summaries link to their corresponding Jenkins builds:

Jenkins Link on Build Summary

Team Services pull requests link to the Jenkins build that validated them:

Jenkins Link on Pull Request

Team Services work items (such as user stories and issues) link to the Jenkins builds in which their associated commits and pull requests were built:

Jenkins Link on Work Item

Links from Jenkins to Team Services

A Jenkins build links to the Team Services build that queued it:

Jenkins Link to Team Services Build

A Jenkins build links to the Team Services branches and commits that were built:

Jenkins Link to Team Services Commits

A Jenkins build links to its associated Team Services pull request and work items (such as user stories and issues):

Jenkins Link to Team Services Pull Request and Work Items

How Can We Improve?

We welcome your suggestions for improving Team Services integration with Jenkins.  We are continuing to enhance it and already have a few ideas:

  • A Jenkins action that queues a Team Services build
  • Jenkins Service Hook support for validating pull requests with a Jenkins build
  • Automatic publishing of test and code coverage results to Team Services when produced by a Jenkins build

The Jenkins Queue Job and Jenkins Download Artifacts tasks are open source.  Contributions are welcome, and you can use these patterns to create your own extensions in the Team Services Marketplace.  To make suggestions or report issues, reach out to us on GitHub.

The Team Foundation Server Plugin for Jenkins is also open source.

For information about Team Services integrations for Java, including plugins for Eclipse, IntelliJ IDEA, and Android Studio, visit java.visualstudio.com.

 

0