Load Agent and Load Controller Installation and Configuration Guide



Microsoft® Visual Studio Team Test Load Agent Products


The Team Test Rig enables Team System users to run tests on one or more remote computers. A rig is made up of a single test controller and one or more test agents. The test controller can be installed on either one or two computers, and like Team Foundation Server, has both application and data tiers. A single test controller is used to coordinate the execution of multiple test agents running tests on multiple computers.


Supported Operating Systems


The following operating systems are supported for the agent and controller:













2008 release


Controller


Windows Server 2008
Windows Server 2003 (SP1)
Windows Vista
Windows XP Professional (SP2)


Agent


Windows Server 2008
Windows Server 2003 (SP1)
Windows Vista
Windows XP Professional (SP2)

 










2005 release


Controller


Windows Server 2003 (RTM)
Windows XP Professional (SP2)


Agent


Windows Server 2003 (RTM)
Windows XP Professional (SP2)
Windows 2000 (SP4)



Hardware Requirements


For the test agent, memory and CPU typically control how much load can be generated. The test controller does not generate load, but does collect load test statistics from the agents and performance counter data from the agents and system under test. Therefore, database used by the controller requires considerable resources both for the amount of data stored and the number of agents involved in the load test scenario. Use the following table as a guide for determining hardware requirements for the controller and agent:


 

















































 


Controller Application Tier


Controller Data Tier


Controller Application/Data Tier


 


Min


Recommended


Min


Recommended


Min


Recommended


CPU


600 Mhz


1 GHz


600 Mhz


1 GHz


600 Mhz


1 GHz


Disk – system


1 GB


1 GB


1 GB


1 GB


1 GB


1 GB


Disk – install


1 GB


8 GB


8 GB


40 GB


8GB


48 GB


Memory


256 MB


512 MB


512 MB


1 GB


512 MB


1 GB


 
























Agent


Minimum


Recommended


CPU


600 MHz


2 GHz


Disk – system


1 GB


1 GB


Disk – install


1 GB


5 GB


Memory


256 MB


1 GB


 


By default, SQL Express is installed with the controller and is used as the default store for load test results. Therefore, the controller runs an instance of SQL Express.  For more information, see “SQL Sizing Considerations” later in this document.


Supported Configurations


Visual Studio, the Test Controller, SQL, and the Test Agent can all reside on the same computer or be installed on different computers. Typical computer configurations are:


Recommended Configuration:


·         Computer 1: Visual Studio


·         Computer 2: Controller, SQL Express


·         Computer 3-n: Agents


Note   Installing an agent on the same computer as the controller can interfere with results collection. If you choose to do this, use the Agent Weighting property to reduce the amount of load generated by the agent installed on the same computer as the controller.


It is recommended that you install the agents on their own computer. Anything that is processing at the same time the agent is running affects the accuracy of the test. Agent weighting reduces the impact, but inaccuracies are still introduced.


The following additional configurations are available. They are listed in order of recommendation.


Alternative Configuration 1


·         Computer 1: Visual Studio, Controller, SQL Express


·         Computer 2-n: Agents


Alternative Configuration 2


·         Computer 1: Visual Studio


·         Computer 2: Controller, Agent


·         Computer 3: SQL Express


·         Computer 4-n: Agents


Alternative Configuration 3


·         Computer 1: Visual Studio


·         Computer 2: Controller, SQL Express, Agent


·         Computer 3-n: Agents


Performance Considerations


Here are some additional considerations to consider when sizing the hardware:


 




























 


Agent


Controller Application Tier


Controller Data Tier


Controller AT/DT


CPU


Depending on the test, the CPU is frequently the limiting factor


Not heavily used


Not heavily used


Not heavily used


Disk


Not heavily used


Not heavily used


10 GB space required for 24 hours of test data


10 GB space required for 24 hours of test data


Memory


Depending on the test, memory will be the limiting factor


Not heavily used


Heavily used by SQL


Heavily used by SQL


 


The amount of load that a specific load agent can generate varies widely from test to test. Most tests are bound by the CPU. CPU use is directly proportional to requests per second (RPS). For other load tests, memory is the limiting factor. The RPS you can expect to drive from a load agent depends upon many factors. These include the following:


·         User load


·         Think time


·         Authentication scheme


·         Size of requests and responses


·         Response time


·         Level of response validation


·         Test Type under load (either Web Test or Unit Test)


Think time is the primary factor for determining the number of users on a CPU-bound test. Changing think times from 2 seconds to 10 seconds allows you simulate 5 times more users, but the RPS being generated will be the same. If your goal is to simulate real users, set the think time to a value that reflects how you believe users will behave on your Web site. Increasing think time and the number virtual users will not necessarily place additional stress on your Web application.


Agents can be bound by memory on tests that use the Connection Per User connection mode. Two connection modes can be configured in the load test run settings. In Connection Pool mode (the default), connections are pooled, but each user still uses two connections when active. In this mode, all virtual users are multiplexed over the connection pool. This allows you to have 1000 active virtual users sharing 100 connections. In Connection Per User mode, each user has a connection that consists of two actual connections open to the server.


If load testing against a typical ASP.NET application with 3 to 5 second think time using Web tests, you can simulate around 1000 users from a single-processor load test agent with a 2 GHz CPU and 1 GB RAM (recommended configuration). The number of users supported is a function of the think time. With longer think times, more users can be supported.


SQL Sizing Considerations


By default, SQL Express is installed on the controller and is used by the controller as the default SQL store for load test results. The SQL Express database is license-limited to store 10 GB of data, which is around 24 hours of load test data for a typical load test. The space that is required for load test data varies greatly depending on the test.


During a load test, samples are collected for each counter instance on each computer. Therefore, the amount of space that is required in the database depends on the following factors: the number of counters collected, the number of computers involved in the test, and the number of samples taken, as controlled by the sample rate.


If appropriate, consider using a separate database to store the load test data. The database can be stored on either the controller computer or on a different computer. To change the data store, submit the SQL commands contained in the .sql file to the SQL server instance you want to use for the load test results store. These are two ways to do this. One way is to use the command sqlcmd from a command prompt and specify the options needed to connect to the desired database. Use the –i option to specify the path to loadtestresultsrepository.sql. Another way is to use one of the GUI interfaces to SQL, such as query analyzer, and open the .sql file and submit the connects.


Test Rig Software Requirements


Setup installs the following required components:


 



















Component


Requirements


Version


Controller


 


SQL Express


2005


.NET Framework


2.0


Agent


.NET Framework


2.0


 


Test Rig User Accounts


The following types of accounts exist in the rig. Before installing, create the required accounts on the rig computers.


 




















Account Name


Description


Requirements


Setup User


User who installs the controller and agent.


This user must be an Administrator on the computer they are running Setup on, and be an Administrator on the Controller.


Controller Service account


User the controller is running as. During Setup, you are prompted for this user account.


This user must have access to read performance counters from computers under test during a load test because the controller service collects performance counters during a load test.


If running in Workgroup (non-domain), a local computer account that has the same user name and password must exist on all agents.


To restart the rig, the Controller Service User is added to the Agent Administrators group during Agent Setup.


Agent Service account


User the agent service is running as. During Setup, you are prompted for this user name.


By default, tests are run as this user. (There is a way to run tests as a different user). Therefore, this user needs the access as required by the tests you are running.


If you turn on Agent Logging, this user needs write access to the installation directory.


 


Requirements for Workgroups


If the computers in a rig are running in Work Group mode, that is, they are not in a domain, or they are running in different non-trusted domains, create local computer accounts on the controller. The accounts should have a matching password for each user who will access the controller, including the Agent service users.


For example, you have three computers, Controller1, Agent1, and Agent2. You must create local computer accounts on each computer for the agent service user and controller user. You also must make sure an administrator account exists on all three computers with the same username and password.


 
































Computer Name


User Account


Password


Controller1


ControllerService


ControllerServicePassword


Agent1


ControllerService


ControllerServicePassword


Agent2


ControllerService


ControllerServicePassword


Controller1


AgentService


AgentServicePassword


Agent1


AgentService


AgentServicePassword


Agent2


AgentService


AgentServicePassword


 


Note: If you are running on Windows XP in a work group, you must turn off Simple File Sharing. See this KB for info on how to do that: http://support.microsoft.com/kb/307874


Open Windows Explorer, choose Tools, and then choose Folder Options, Next, select the View tab, and turn off the “Use Simple File Sharing (Recommended)” option.


Accessing the Controller


After the controller is installed, access to the controller is limited to members of the TeamTestControllerUsers and TeamTestControllerAdmins groups that were created during Setup, and to the Administrators group. Add appropriate users and/or groups to these groups to allow them to access the controller. Members of the TeamTestControllerAdmins group or the Administrators group can administer the controller by clicking the Test menu, and then choosing Administer Test Controller in Visual Studio. Members of the TeamTestControllerAdmins group must also be power users or administrators on the controller computer.


Team Test Load Controller Installation


Warning   Set up the Controller before setting up any agents.


Note   Load controllers cannot be installed on domain controllers.


1.      Start setup.exe located in the vs\controller folder.


The Welcome to Setup page appears.


2.      Click Next.


The End-User License Agreement and Product Key page appears.


3.      Review the End-User License Agreement (EULA) and select I accept the terms of the license agreement, if appropriate.


4.      Enter the product key information and then click Next.


The Destination Folder page appears.


5.      Click Browse to select a different destination folder, or click Next to use the default folder.


The Service Account page appears.


6.      Enter the account name and password for the controller service account and then click Next.


The Ready to Install page appears.


7.      Review the installation information and then click Install to continue.


The Installing Components page appears.


8.      Upon successful installation, the Setup Completed Successfully page appears. Click Finish to complete the Setup program.


During Setup, you are prompted for the controller service account. The controller service account collects performance counters on other computers during a load test. Use a domain account that can easily be granted access to many computers. Grant the controller user rights to monitor performance on all the systems under test for the built-in performance monitoring capabilities to work.


Setup creates three local groups:


·         TeamTestControllerAdmins


·         TeamTestControllerUsers


·         TeamTestAgentService


Add users who run tests to TeamTestControllerUsers. Add users who administer the test rig to TeamTestControllerAdmins. Agent setup automatically adds the users who Agent Services run as to TeamTestAgentService.


Note   The user running the controller setup must be a member of local Administrators group.


The controller runs as a Windows Service on the computers on which it is installed. To view the properties of the service use Control Panel. In Control Panel, choose Administrative Tools and then choose Services. The name of the controller service is the Visual Studio Team Test Controller. The controller services can be started and stopped like other services in Windows.


The controller and agents can be configured and monitored by using the Administer Test Controller dialog box, available on the Test menu. For more information, see “How to: Administer the Rig” in the product documentation.


By default, the controller uses SQL Express to store load test results. SQL Express is license limited to store 4GB of data, which is approximately 10 hours of load test data. If you choose to use another SQL database to store the results, you can set up the Load Test Results Store after you have installed the controller. For more information, see “How to: Configure a Results Store using SQL” in the product documentation.


After you have configured your Load Test Results Store, you must indicate to the controller where the results are to be stored. For more information, see “How to: Identify a Load Test Results Store in the product documentation.


Load Controller Firewall Exceptions


If you are running Windows Firewall, the controller setup adds the following exceptions to your firewall:


 



















Exception


Reason


QTControllerService.exe


Enable communication with the controller


SQLExpr.exe


Enable Visual Studio Load Test Results Analyzer to access load test data stored in SQL Express


SQLBrowser.exe


Enable Visual Studio Load Test Results Analyzer to access load test data stored in SQL Express


File and Printer Sharing TCP 139


Enable Restart Rig in Administer Test Controllers dialog to work.


Enable Agent setup to automatically add the Agent service user accounts to the TeamTestAgentService user group.


Warning   Setup does not add this firewall exception. Add this exception manually.


 


If you are running on Windows XP in a work group, you must turn off Simple File Sharing. See this KB for info on how to do that: http://support.microsoft.com/kb/307874


Team Test Load Agent Installation


Warning    Set up the Controller before setting up any agents.


1.      Start setup.exe located in the vs\agent folder.


The Welcome to Setup page appears.


2.      Click Next.


The End-User License Agreement and Product Key page appears.


3.      Review the End-User License Agreement (EULA) and accept the terms of the license agreement, if appropriate.


4.      Enter the product key information and then click Next.


The Destination Folder page appears.


5.      Click Browse to select a different destination folder, or click Next to use the default folder.


Note   If you are installing the Agent on the same computer as the Controller, the option to choose a new location is disabled.


The Specify Controller page appears.


6.      Enter the computer name of the Team Test Load Controller computer and click Next.


Note   You can locate the name of the controller computer by clicking My Computer, right-clicking, choosing Properties and viewing the Computer Name tab.


The Ready to Install page appears.


7.      Review the installation information and then click Install to continue.


The Installing Components page appears.


8.      Upon successful installation, the Setup Completed Successfully page appears. Click Finish to complete the setup.


Following this, you will be prompted to restart your computer to complete the installation.


Note   The user running the agent setup must be a member of local Administrators group and a member of the controller’s Administrators group.


During setup, you are prompted for the name of the controller computer that this agent will be associated with. Agent setup communicates with the controller to add the agent to the controller.


During setup, you are prompted for the Agent user. This user is automatically added to TeamTestAgentService group on the controller. Only Agent Service Users who are a member of this group are allowed to communicate with the controller.


The Agent service is named Visual Studio Team Test Agent. To view the properties of the service use Control Panel. In Control Panel, choose Administrative Tools and then choose Services. The agent services can be started and stopped like other services in Windows. Also the agents can be administered using the Administer Test Controller dialog box, available on the Test menu. For more information, see “How to: Administer the Rig” in the product documentation.


Note   AgentConfigUtil.exe is a command line utility that can be used to assign an Agent to a different Controller, or to add an Agent to a Controller if Agent setup fails to do so.


Load Agent Firewall Exceptions


If you are running Windows Firewall, Agent setup added the following exceptions to your firewall:


 






















Exception


Reason


QTAgentService.exe


Enable communication with the controller


AgentServiceUI.exe


Enable communication with the controller


QTAgent.exe


Enable communication with the controller


AgentConfigUtil.exe


Enable communication with the controller


File and Printer Sharing Port 139


Enable Restart Rig in Administer Test Controllers dialog to work


Enable collection of performance counters


Warning   Setup does not add this firewall exception. Add this exception manually.


 


If you are running on Windows XP in a work group, you will also need to turn off Simple File Sharing.


Troubleshooting


Note   To verify the installation, run Visual Studio and launch the Administer Test Controller dialog box (available from the Test menu), connect to the controller, and check the status of the agents. Agents should be in the Ready state. If an agent is not in the list of agents, reinstall the agent or run AgentConfigUtil.exe after verifying that the controller is installed correctly. If the agent is in the Disconnected state, check that the agent service user account has been added to the TeamTestAgentService user group on the controller. Also check the <installdir>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\agentConfigUtilSetup*.log file for errors.


“Rig failed to restart for controller <controller computer name>. The following error was reported Cannot open VSTTController service on computer <controller computer name>.”


This error occurs when a user tries to restart the rig from the Administer Test Controller dialog box, but while this user has been added to the TeamTestControllerAdmins group, they have not also been added to the Power User or Administrator group.

Comments (23)

  1. Here is the install guide for the agent and controller. It contains a bunch of useful sizing and configuration

  2. MSDNArchive says:

    Hi Ed,

    I am trying to set up an environment where I will have a test controller communicating with agents through NAT. The agents will work in workgroup mode as you specify. However I need to know two things to verify if my approach will work:

    1. Do TCP connections only go from Controller to Agent, or does sometimes an Agent initiate communication with the Controller? (If yes, then I would need to work around it in NAT config somehow)

    2. What exact ports need to be opened between the test controller and agents? You only specify that there are some exceptions, but do not give info what exact exceptions are needed. I need this information to set up the firewall rules that isolate the test controller and test agents.

    Another approach that I am considering is placing the Test controller on the same network as agents, but then I have the issue of Test Controller <-> TFS communication. Do you know if it is possible to run such configuration if the test controller is in one domain and the TFS server in another untrusted domain?

    Which approach would you recommend?

    The general problem is that the development environment where the TFS server sits is another domain and another network separated by NAT and firewall from the test environment. I would imagine that is a pretty common scenario and there must be a solution to that.

    Thanks in advance for any information,

    Andrzej

  3. JMarkHowell says:

    Ed, thank you SO much.   This is exactly what I was looking for.  Keep up the great work!

    Mark.

  4. nileshgujarathi says:

    Hi Ed

    Thanks for the Agent and controller help.

    I have downloaded visual studio team system  2005 180 days trail version.

    and now I want to install Agent and controller.

    As you mentioned that setup files are present in the folder vsAgent and VSController.

    But In the VSTS setup folder i have not found the vsagent folder and vscontroller folder

    so please explain me from where i will find this folder for the vsts Agent and Controller installation.

    Also explain me that VSTS 2005 180 days trial version supports agent and Controller functionality.?

    Thanks

    Nilesh

  5. In my previous blog article, I pointed out that in a workgroup environment, Windows XP has the force

  6. In my previous blog article, I pointed out that in a workgroup environment, Windows XP has the force

  7. Shai Raiten says:

    Load Agent and Load Controller Configuration Guide Published On: Ed Glas&#39;s blog The Team Test Rig

  8. Shai Raiten says:

    Load Agent and Load Controller Installation Guide Published On: Ed Glas&#39;s blog Team Test Load Controller

  9. [ Nacsa Sándor , 2009. február 8.] Ennek a processzoronként licencelt szoftvernek a segítségével skálázni

  10. DSF says:

    Hi Ed.

    We have a setup where the controller is within the test domain A and the load agent has 2 NIC cards to access domain A while on domain B. The test rig thus has the controller in domain A while it controls the load agent which is in domain A&B.

    The interesting thing that happens here is that if you initiate a load test from the load agent side, the test rig shows all is connected but it remains in a pending state and does not start the actual load test. When you initiate the same test from the Controller the test rig starts the tests with no problem.

    We have narrowed the problem down to what we think the problem is and that is the fact that we have not setup Kerberos delegation in our environment and initiating the test from the load agent side means a hop from the agent to the controller and than back to the load agent again, where starting the test from the controller is only one hop from the ctlr to the agent. Please comment on this theory, as we don’t find any error logs or exceptions to prove that. Also are there any other issues or tips you can recommend to solve this other than setting up Kerberos or moving the load agent onto domain A only ?

    Regards.

    David.

  11. jpt007 says:

    I cannot seem to find what is the recommended # of NIC’s for an Agent?

  12. Santhan says:

    Hi please advise

    I want to know if the Service Account and password that we specify for the controller and at the agent:

    1. during the agent installation does the controller box needs to be logged in as this service account?

    2. I am running the agent on my box and my project lead is running the controller on her box. Please advise how the accounts shoud work (we are in the same domain)??

    thanks

    cheers

    San

  13. Santhan says:

    I meant

    I want to know if the Service Account and password that we specify for the controller and at the agent should be the account used when installing the  controller setup at one box and the agent setup at another box.

  14. DH_1313 says:

    I have setup the controller and a remote agent.  There is a firewall in between with the required ports open but the agent is still showing as disconnected.  These are two different domains and a trust is not going to be allowed.  All user accounts have been setup as local users and all match with passwords.  Please help as this is delaying testing.

    Thanks

  15. Srivani says:

    According to given installation procedure this tool accepts only .Net framework with version 2.0.

    does it accepts the versions also. i want to know whether it accepts .Net 3.5 also

  16. AB says:

    Hi

    I have installed VS Ultimate -2010, and could someone guide how to install a controller and UserAgents using Ultimate.

  17. ASdf says:

    Hi

    Installed VS Ultimate 2010.  Could someone help how to install Controller and Vu Gens. on top of Ultimate? or I need to uninstall ultimate and install VSTS.  Please guide.

    -ASdf

  18. MarkinWales says:

    Is it possibe to have both a 2007 agent and a 2010 agent on the same computer?  Doing this will help keep our VMWare deployment as small as possible.

  19. TestArch says:

    While configuiring Agents for Load testing what option is recommended amongts the both:

    1. Run the Test agen as

    "Service"

    OR

    " Interactive process"

    I am in middle of configuring the agents. it will be great if you can let me know the suitable option to select.

  20. Hey!! This is really good..

    Could you come up with such a requirement for Visual Studi 2010 as well.

  21. Gaurav Harode says:

    Hey ED !!,

    i want to know that how to add the Agent computers while prepareing to perform the Load Testing in VSTS 2008.

  22. Anonymous says:

    Please tell me how i install Agent and controller visual studio 2008 into my system, send me the link which i install everything..