Running Web and Load tests from the Command-Line


This blog post will show you how to run a web test or load test from the command-line instead of from within Visual Studio.  Most of the time you will want to run you web or load test from within visual studio, but you can run them from a command-line.  Maybe you have a set of 5 web tests that you want to run once a day.  You could configure a scheduled task to run a batch file which contained ran the 5 web tests.  The command line executable that is used to run tests is called mstest.exe and is installed when you install either VS Team Suite or VS Team Test SKU.  Here is a MSDN help link for mstest:  http://msdn.microsoft.com/en-us/library/ms182486.aspx


 


The easiest way to get started with mstest is to start a visual studio command prompt.  Go to Start Menu ->  All programs -> Microsoft Visual Studio v9.0 -> Visual Studio Tools -> Visual Studio 9.0 Command Prompt.   This will launch a command window which has mstest.exe in its path.  So type mstest /?  This will show each of the available options.  Let’s run through a few options of how to run tests.


1)      Running a single WebTest


First let’s run a simple web test.  To do this, we just need to specify the /TestContainer argument.  A .webtest file or .loadtest file is considered a test container in the same way that a dll which contains unit tests is a test container.  So if we have a web test called WebTest1.webtest, you would use the following command to run the web test:


mstest /TestContainer:WebTest1.webtest


This will produce output similar to the following:


Loading WebTest1.webtest…


Starting execution…


Results               Top Level Tests
——-               —————
Passed                webtest1.webtest


1/1 test(s) Passed


Summary
——-
Test Run Completed.


  Passed  1
 ———
  Total   1


Results file:  C:\Documents and Settings\slumley\My Documents\Visual Studio 10\Projects\TestProject3\TestProject3\TestResults\slumley_SLUMLEYTCM 2008-12-22 13_38_14.trx


2)      Running Multiple WebTests


Now let’s execute 2 web tests from one call to mstest.  You can specify multiple test container arguments on one command-line.  So say we wanted to execute WebTest1.webtest and WebTest2.webtest.  You would use the following command-line:


 


mstest /TestContainer:WebTest1.webtest /TestContainer:WebTest2.webtest


 


This will produce output similar to the following:


 


Loading WebTest1.webtest…


Loading WebTest2.webtest…


Starting execution…


 


Results               Top Level Tests


——-               —————


Passed                webtest1.webtest


Passed                webtest2.webtest


2/2 test(s) Passed


 


Summary


——-


Test Run Completed.


  Passed  2


  ———


  Total   2


Results file:  C:\Documents and Settings\slumley\My Documents\Visual Studio 10\P


rojects\TestProject3\TestProject3\TestResults\slumley_SLUMLEYTCM 2008-12-22 13_4


3_04.trx


 


 


3)      Specifying deployment items


One of the big differences with running from the command line is that you are not within a visual studio project.  When you run a web test from within Visual Studio, we will try to determine what needs to be deployed with the web test in order for it to run.  For example, maybe you have written a custom validation rule or a extraction rule.  When you run the test from within visual studio, we will look at the references of the test project and try to figure out what needs to be deployed. 


 


When you are running from the command-line, you will need to be very explicit about what you deploy.  So if you have a dll that needs to be deployed for a test to run, you will need to inform mstest.  The way you do this is by specifying /runconfig parameter.  One of the sections of a run config is deployment items.  Use visual studio to create a run config.  Then to run a test with a runconfig, you would have a command-line similar to the following:


 


Mstest /TestContainer:WebTest1.webtest /RunConfig:NewRunConfig.testrunconfig


 


4)      Executing  a test on a controller/agent setup


When in VS, to run a web or load test on a controller/agent setup you need to use a test run configuration.  You would open the run config, specify to run the test remotely and specify a controller name.  It works the same way from the command-line.  You will need to create a run config that has the correct settings.  Then you run it the same way as you do when specifying deployment items:


Mstest /TestContainer:WebTest1.webtest /RunConfig:NewRunConfig.testrunconfig


 


5)      Running a coded web test


Running a coded web test is similar to running a unit test.  You will set the /testcontainer argument to the dll which contains the coded test.


 


mstest /TestContainer:TestProject1.dll


 


One thing you will notice when running a test this way, is that mstest will execute all of the tests within the dll.


 


6)      Executing a specific coded test in a dll


So if you want to execute just one test within a dll, you would need to use the /test argument.  For example, to run WebTest1Coded, you would use the following command line:


 


mstest /TestContainer:TestProject1.dll /Test:WebTest1Coded


 


If you wanted to execute 2 tests, you can specify multiple /Test arguments.  For example:


 


mstest /TestContainer:TestProject1.dll /Test:WebTest1Coded /Test:WebTest2Coded


 


7)      Executing a load test


This is the same as executing a web test except you need to specify the .loadtest file


mstest /TestContainer:LoadTest1.loadtest


When the test completes, you can either open the trx file to get to the load test results or you can use the manage load test results dialog form within VS.  Here is a blog on that: http://blogs.msdn.com/slumley/pages/managing-load-test-results.aspx


 


8)      Specifying the results file name


You will notice that the results file(trx file) gets a unique name which contains user, machine and a timestamp.  If you want to specify the name of the results file and where it is generated, you can use the /resultsfile parameter.  For example, to create a results file called MyResults.trx and have it saved to c:\results, I would do the following:


 


mstest /TestContainer:WebTest1.webtest /resultsfile:c:\results\MyResults.trx


 


Hopefully this post shows you how to make use of the mstest tool.


 


 

Comments (3)

  1. This blog post will show you how you can run web and load tests from the command line: http://blogs.msdn.com/slumley/pages/running-web-and-load-tests-from-the-command-line.asp

  2. This blog just summarises some of the issues I picked up … some of which ‘again’ … during discussions

  3. This is a very helpful blog. How about the /searchpathroot switch? Team foundation build passes this switch to MSBuild.exe, while this switch is not documented.

Skip to main content