Explanation of V.S. Load Test warm-up, load patterns, and cool-down, and how this affects the reported results

These graphs show what test iterations are stored and reported in the results of a Visual Studio load test based on the load pattern and warmup used. It is important to note the following things when considering the results:

  1. If the test's active run settings is set to store "All Individual Details" then all iterations are stored in the various "Details" tables regardless of whether they are reported.
  2. In the case of #1 above, the various "Details" tables have a column called "InMeasurementInterval." The iterations shown in Blue will have the value set to 1 and the iterations in Yellow will have a value of 0. Iterations in Red are not recorded.
  3. If the test's active run settings is set to store "Statistics Only" then the "Details" tables will NOT contain any information for the run in question.
  4. The summary information shown in the tables of a load test result page are calculated from the run details at the end of the run and are stored in Summary tables. These numbers will NOT include any of the iterations shown in Yellow below, regardless of which storage detail level was chosen.

The two graphs show two different test runs that drove the exact same load against the System Under Test.

  1. Each horizontal line (blue and yellow) represents a vUser executing a single iteration of the web or unit test. The Blue lines are considered "In the measurement interval" and are used for reporting. The yellow are not.
  2. The red line indicates the "apparent user load", meaning the load that the S.U.T. is experiencing. NOTE that the scale of the red line here is not exact. It is for illustration purposes only).
  3. The vertical dashed lines indicate the points in time that the test harness (Visual Studio) implements a "change of state" for the test.
    1. The first graph shows a run with a warmup time (first State), A Constant Load of 4 vUsers (second State), a cooldown time (third State), and the end of execution (any iteration in flight at the end of cooldown is terminated in flight and the results are not kept at all (shown in red below).

     

Graph showing a test with warmup and a constant load pattern. All iterations in blue will be reported. All iterations in yellow will not be reported.

   

Graph showing a test with no warmup and a step load pattern. All iterations in blue will be reported. All iterations in yellow will not be reported.