Understanding Build and Release Pipelines - Visual Studio Team Services/Team Foundation Server.

  • Article

This article is provided in a FAQ format, based on our experience with customers to explain the concept of Pipelines. Hope it helps!

Why are Pipelines necessary?

  • To run concurrent builds and/or deploy concurrent releases.
  • The number of pipelines is the maximum number of concurrent builds you can run and/or releases you can deploy at any point in time.

Types of Pipeline:

  • Private pipelines
    • VSTS - run multiple builds and/or deploy multiple releases
    • TFS 2017 - deploy multiple releases
    • With Private Pipelines, you install agent software supplied by Microsoft on machines that you run and manage.
  • Hosted pipelines
    • Only for VSTS - run multiple builds and/or deploy multiple releases simultaneously using a pool of agents managed by Microsoft.
    • Microsoft is responsible for patching and updating these machines so you don’t need to worry about this.

Private pipelines:

  • There is one Private Pipeline included with each Team Services account
  • Plus one additional Private Pipeline for each active Visual Studio Enterprise subscriber who is a member of your account.
  • With TFS, you only need pipelines for deploying releases; no pipelines are required for builds since unlimited concurrent builds are included with the TFS server license.
  • For Release in TFS 2017: One Private Pipeline is included with TFS 2017, plus one additional Private Pipeline for each active Visual Studio Enterprise subscriber within your organization.
  • Do I need concurrent pipelines in TFS 2015 or TFS 2013? Short answer: NO.

Hosted pipelines:

  • With your VSTS account, you start with 240 minutes on one Hosted Pipeline for free, with a maximum single job duration of 30 minutes.
  • With the first purchase of a hosted pipeline, the 240 minute limit on total build and release time as well as the 30 minute limit on a single job are waived. It increases the maximum duration of a single job to 6 hours.
  • Each additional purchase of a hosted pipeline adds another hosted agent for running your builds and releases.

Resource limits:

To view the number of pipelines purchased, open the Build and Release section of the Settings page, and select Resource limits.

VSTS: TFS 2017: What if a user doesn’t want to Purchase extra Hosted pipeline, but the free 240 minutes are over?

Open Azure portal then navigate to -> Team Services accounts -> Select the VSTS account -> Build and Deployment service

Change the minutes to PAID and set the limit.

 

Build per minute pricing is…

*First 240 minutes per month free

*$.05/minute for the next 20 hours/month

*$.01/minute for usage above 20 hours/month

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Is sharing of pipeline (Private and hosted) is possible?

  • You cannot use private pipeline for hosted agent.
  • You can use hosted pipeline for private agent.
    • However…

Hosted pipeline can only reach out to machines that are publicly addressable.

They cannot talk to machines that are not addressable from the Internet.

They cannot establish a ExpressRoute connection to your machines either.

So, if your on-premise machine is behind your firewall and cannot be reached publicly, then you need a private pipeline.

 

How a pipeline is consumed:

Consider a VSTS account that has only one pipeline.

  1. FabrikamFiber CI Build 102 (master branch) is first to be started.
  2. Deployment of FabrikamFiber Release 11 is triggered by completion of FabrikamFiber CI Build 102.
  3. FabrikamFiber CI Build 101 (feature branch) is triggered. The build can't start yet because Release 11's deployment is active. So the build stays queued.
  4. Release 11 waits for approvals. Fabrikam CI Build 101 starts because a release waiting for approvals does not consume a pipeline.
  5. Release 11 is approved. It resumes only after Fabrikam CI Build 101 is completed.

A release requires a pipeline only when it is being actively deployed to an environment. Waiting for an approval or a manual intervention does not consume a pipeline.

Concurrent processing within a single build or release:

  • Concurrent processing within a single build or release does not require additional concurrent pipelines. So long as you have enough agents, you can:
    • In a build process, run multiple build configurations at the same time.
    • In a release process, deploy to multiple environments at the same time.

Example:

Suppose your VSTS account has three concurrent pipelines. You can have more than three Private agents running at the same time to perform parallel operations within builds and releases.

Sharing of pipelines across projects in a collection:

  • Pipelines are purchased at the account level, and they are shared amongst all projects in an account.
  • Cannot partition or dedicate certain pipelines to a specific project or agent pool.
  • For Example:
    • You purchase two pipelines in your account.
    • You queue two builds in the first project, and both the pipelines are consumed.
    • You queue a build in the second project. That build will not start until one of the builds in your first project is completed.

References:

Content: Kirthish Kotekar Tharanatha
Reviewer: Deepak Mittal