Using a load test plugin to limit the number of tests run in a load test

Normally the number of iterations of the inner tests (usually web tests and unit tests) run during a Visual Studio Team System (VSTS) load test is determined by the duration of the load test.  However, some users which to control the number of inner test iterations exactly.   There is not direct support for this in VSTS, but it can be accomplished using a load test plugin.   Below is an example of a load test plug-in that does this.  Instructions for using it are included in the comments.

Note that if the load test is run on a test rig with more than one load test agent, the number of test iterations specified will be run on each of the agents.  You can adjust for this by specifying a smaller number of tests for the value of the "MaxTestIterations" context parameter, or modify the load test plug-in code to divide that number by the number of agents (but I'll leave that as an excercise for the reader).

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using Microsoft.VisualStudio.TestTools.LoadTesting;

namespace Plugins
{
    /*
       This class is a sample load test plugin that allows you
       to limit the number of test iterations that are run during a load test.
      
        To use this class:

        1. Add the class to a test project.
        2. Build the test project.
        3. Hook this load test plugin up to your load test:

            * In the load test editor, select the top-most node in the load test tree.
              This is the load test name in the tree.
            * Go to the property sheet.
            * Click on the symbol next to the value for "Load Test Plug-in".
            * Select the MaxTestIterationsLoadTestPlugin class as your plugin.
     
        4. Add a context parameter named "MaxTestIterations" to your load test:
        
            * Right-click the "Active Run Settings" node in the tree
            * Select "Add Context Parameter" - you will see the property sheet
            * For the parameter name, type "MaxTestIterations"
            * For the parameter value, specify the maximum number of tests to run
              Note that the duration specified for the load test still applies; the
              load test will stop when the time specified for the duration has expired
              even if the number of tests executed is less than "MaxTestIterations".
    
        5. (Optional) Add a context parameter named "AbortWhenMaxTestsCompleted" to your load test:
            * Follow the same step as for #4 above, but type "AbortWhenMaxTestsCompleted" for the
              context parameter name, and either "true" or "false" for the value:
                - When true, the load test plug-in will abort the load test as soon as the number of
                  test specified by MaxTestIterations has been completed.   The value for "Outcome"
                  in the Visual Studio Test Results window will be "Error" in this case.
                - When false, the load test will continue to run until the duration specified for
                  the load test is complete even after there are no more tests running (so "Tests / Sec"
                  and "Requests / Sec" will drop to 0 for the remainder of the test.  This is the default.    
     */
    public class MaxTestIterationsLoadTestPlugin : ILoadTestPlugin
    {
        #region ILoadTestPlugin Members

        public void Initialize(LoadTest loadTest)
        {
            m_loadTest = loadTest;
            if (m_loadTest.Context.ContainsKey("MaxTestIterations"))
            {
                try
                {
                    m_maxTestIterations = int.Parse((string)m_loadTest.Context["MaxTestIterations"]);
                }
                catch (FormatException)
                {
                    throw new ApplicationException("MaxTestIterations not in integer format");
                }
                m_loadTest.TestStarting += new EventHandler<TestStartingEventArgs>(m_loadTest_TestStarting);

                if (m_loadTest.Context.ContainsKey("AbortWhenMaxTestsCompleted"))
                {
                    try
                    {
                        if (bool.Parse((string)m_loadTest.Context["AbortWhenMaxTestsCompleted"]))
                        {
                            m_loadTest.TestFinished += new EventHandler<TestFinishedEventArgs>(m_loadTest_TestFinished);
                        }
                    }
                    catch (FormatException)
                    {
                        throw new ApplicationException("AbortWhenMaxTestsCompleted not in bool format");
                    }

                }
            }
        }

        void m_loadTest_TestStarting(object sender, TestStartingEventArgs e)
        {
            if (++m_testsStarted == m_maxTestIterations)
            {
                SetLoadForAllScenariosToZero();
            }
        }

        void SetLoadForAllScenariosToZero()
        {
            foreach (LoadTestScenario scenario in m_loadTest.Scenarios)
            {
                scenario.CurrentLoad = 0;
            }
        }

        void m_loadTest_TestFinished(object sender, TestFinishedEventArgs e)
        {
            if (++m_testsFinished == m_maxTestIterations)
            {
                m_loadTest.Abort();
            }
        }

        private LoadTest m_loadTest;
        private int m_maxTestIterations;
        private int m_testsStarted;
        private int m_testsFinished;

        #endregion
    }
}