Web/Load Test Automation

You can use MsTest.exe to execute Unit, Web, and Load tests from a command line. For more information, please read https://msdn.microsoft.com/en-us/library/ms182486.aspx. Sean Lumley also blogged the details on how to run Web, coded Web, and load tests through mstest.exe. This post will show you by creating a wrapper of the mstest process, you can automate Web/Load tests in code.

Step 1 – Create a RunHelper class to wrap up the MsTest.exe process

This helper class constructs the command-line argument list for MsTest.exe, starts the MsTest process to run your tests, and logs the output. You can modify the MSTestArgumentsFormat to use different set of mstest switches.

public class RunHelper

{

    static readonly string VSPath = Path.Combine(Environment.GetEnvironmentVariable("ProgramFiles"),

                                                  @"Microsoft Visual Studio 10.0\Common7\IDE");

    static readonly string MSTestPath = Path.Combine(VSPath, "MSTest.exe");

    static readonly string MSTestArgumentsFormat =

        string.Format(CultureInfo.InvariantCulture,

                "/testcontainer:{0} /testsettings:{1} /resultsfile:{2}",

                @"{0}",

                @"{1}",

                @"{2}"

            );

    public static void RunMSTest(string container, string testsettings, string resultsFileName, int timeout)

    {

        // Build the command line argument

        string commandLineArguments = string.Format(CultureInfo.InvariantCulture,

                                            MSTestArgumentsFormat,

                                            container,

                                            testsettings,

                            resultsFileName

                                        );

        ProcessStartInfo startInfo =

            new ProcessStartInfo(

                    MSTestPath,

                    commandLineArguments

                );

        startInfo.CreateNoWindow = true;

        startInfo.UseShellExecute = false;

        startInfo.RedirectStandardError = true;

        startInfo.RedirectStandardOutput = true;

        startInfo.WindowStyle = ProcessWindowStyle.Hidden;

        // Create and run the process

        Process mstestProcess = Process.Start(startInfo);

        mstestProcess.StartInfo = startInfo;

        mstestProcess.Start();

        string stdout = mstestProcess.StandardOutput.ReadToEnd();

        string stderr = mstestProcess.StandardError.ReadToEnd();

        mstestProcess.WaitForExit(timeout);

        int exitCode = mstestProcess.ExitCode;

        mstestProcess.Close();

        // log the output

        Console.WriteLine("{0} {1}", MSTestPath, commandLineArguments);

        Console.WriteLine("exitcode: {0}", exitCode);

        Console.WriteLine("stdout: {0}", stdout);

        Console.WriteLine("stderr: {0}", stderr);

    }

}

Step 2 – Create a TestRunner class to run all the tests

I added three simple functions to the following helper class for running Web, coded Web, and load tests. Basically, you can add your logics to the methods to use different testsettings files in different test environments, e.g. local vs remote, 32-bit mode vs 64-bit mode, running tests with or without data collection, and running tests using different controllers.

public class TestRunner

{

    // Specify a proper timeout for you testing

    static readonly int TIMEOUT = 120000;

    static public void RunWebTests()

    {

        RunHelper.RunMSTest(@"e:\webtest1.webtest",

                            @"e:\local.testsettings",

                            @"e:\webtest1Result.trx",

                            TIMEOUT);

    }

    static public void RunCodedWebTests()

    {

        RunHelper.RunMSTest(@"e:\codedTests.dll",

                            @"e:\remote.testsettings",

                            @"e:\CodedWebtestResults.trx",

                            TIMEOUT);

    }

    static public void RunLoadTests()

    {

        RunHelper.RunMSTest(@"e:\Stress1.loadtest",

                            @"e:\64Bit.testsettings",

                            @"e:\Stress1Result.trx",

                            TIMEOUT);

    }

}

Step 3 – Run the tests from a console application

The last step is to create a console application, add classes created in step 1 and 2, and call methods in the TestRunner class. In the following code, it just calls the TestRunner to run all the Web tests.

static void Main(string[] args)

{

    TestRunner.RunWebTests();

}

And the output is

E:\school\TestProject1\ConsoleApplication1\bin\Debug>ConsoleApplication1.exe
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /testcontainer:e:\webtest
1.webtest /testsettings:e:\local.testsettings /resultsfile:e:\webtest1Result.trx
exitcode: 0
stdout: Microsoft (R) Test Execution Command Line Tool Version 10.0.30208.0
Copyright (c) Microsoft Corporation. All rights reserved.

Loading e:\local.testsettings...
Loading e:\webtest1.webtest...
Starting execution...

Results Top Level Tests
------- ---------------
Passed e:\webtest1.webtest
1/1 test(s) Passed

Summary
-------
Test Run Completed.
Passed 1
---------
Total 1
Results file: e:\webtest1Result.trx
Test Settings: Perf Run

stderr: