Test Controller, Test Agent versions and their usage in different scenarios

There have been several questions on the different scenarios related to running automated tests in various workflows and which versions can / cannot be used.

Let’s use the following 2 names for categorizing how agents get used:

  1. Test Controller/Test Agent (TC/TA) scenarios – this refers to on-premises load testing, running automated tests in the XAML build workflows, triggering an on-demand execution of automated tests using MTM and creating and managing lab environments using Visual Studio Lab Management.
  2. Team Build / Release Management scenarios – this refers to running functional tests in the Team Build / Release Management workflows, using the ‘Visual Studio Test Agent Deployment’ and ‘Run Functional Tests’ tasks from the task catalog.

Agents for Visual Studio 2013 Update 5

  • Can be used for TC/TA scenarios
  • Cannot be used for Team Build/Release Management scenarios
  • Can run tests that were built using Visual Studio 2013 and Visual Studio 2015.

Agents for Visual Studio 2015 and Agents for Visual Studio 2015 Update 3

  • Cannot be used for TC/TA scenarios
  • Can be used for Team Build / Release Management scenarios
  • Can run tests that were built using Visual Studio 2015 and Visual Studio 2013.

Agents for Visual Studio 2017

  • Can be used for TC/TA scenarios
  • Can be used for Team Build / Release Management scenarios.
  • Can run tests that were built using Visual Studio 2017.

Offline installers can be created using the steps similar to the Visual Studio offline installer, described here.

So, for example, to create an offline installer for Test Agent:

  1. First download the setup bootstrapper (vs_testagent.exe)
  2. To create the offline installation, run “vs_testagent.exe –layout c:\VSTestAgent2017Offline” from the directory where vs_testagent.exe got downloaded to.

For options to customize the offline installer, refer to the doc.

TFS 2017

TFS 2017 RTM – TFS 2017 RTM shipped with ‘Visual Studio Test Agent Deployment’ (v1) and ‘Run Functional Tests’ tasks that can be used in the Team Build / Release Management scenarios. These tasks use the Agents for Visual Studio 2015 Update 3 package. As such, the tasks in their current form can be used for running tests built using Visual Studio 2015 and Visual Studio 2013. They cannot run tests built using Visual Studio 2017.

TFS 2017 Update 1 – TFS 2017 Update 1 shipped with ‘Visual Studio Test Agent Deployment’ (v2) and ‘Run Functional Tests’ tasks that can be used in the Team Build / Release Management scenarios. The test agent deployment task has the ability to specify which Agent version (2015 or 2017) should be deployed. Based on the selection, the appropriate agent will get deployed and can run tests built using Visual Studio versions as described in the previous sections.

 

9