VSTS 2010 Feature: More flexible load modeling via new options on Load Test Scenarios

  • Article

In VSTS 2010 Beta 1 (available for download here), there are four new properties on each Scenario in a Load Test that give you more control of the time, location, and duration of the running of the Scenario during the load test. You can access the Scenario property sheet by right-clicking on the desired Scenario’s node in the Load Test Editor tree:

GettingToScenarioProperties

The new properties are:

“Agents to Use”
”Maximum Test Iterations”
”Delay Start Time”
”Disable During Warmup”

The screen shot below of the Scenario property sheet shows the new Scenario properties along with those that existed in VSTS 2008:

NewScenarioPropertiesDefaults

The above screen shot shows the initial default values for all four properties which in all four cases means that the new feature is not used and the load test behaves as it would with VSTS 2008 (all other things being the same).

The screen shots below show non-default values for each of these new properties. You can read the property descriptions in the screen shot, so no need to repeat those, but I’ve added some additional comments on the selected property below each screen shot:

NewScenarioPropertiesNonDefault 

The agent names that are entered should be the names of agents that are connected to the controller to which the load test will be submitted. They should be the simple computer names of the agents (as seen in the “Computer Name” field in the Control Panel). Unfortunately, at this time, if you switch to submitting the load test to a different controller, you will need to change the value for “Agents to Use” as there is no way to parameterize this list to vary depending on the controller used. This list of agents designates a subset of those the agents that are connected to the controller, and are in the Ready state when the load tests starts (they may be running a different load test or other test run when the load test is queued as long as they become Ready when the load test is taken out of the Pending state and starts running), and that meet any agent selection criteria to allow the test run to be run on the agent. The Scenario will run on all agents in the list that meet these criteria, and the user load for the Scenario will be distributed among these agents either evenly (by default) or according to any agent weightings specified in the Agent properties for the agents (from the “Administer Test Controllers” dialog in Visual Studio).

The screen shot below has a non-default value for “Maximum Test Iterations” which is selected to show the property description:

NewScenarioPropertiesMaxIterations L

This property is a maximum on the total number of test iterations for tests in the selected Scenario for all users on all agents on which the Scenario is run, and does not mean “Test Iterations per User”. If the Load Test Run Settings dialog specifies a value for Test Iterations that is less than this value, or the timed duration of the load test expires before this maximum is reached the load test will stop. In other words, as the name implies this is a maximum, but not a minimum.

The screen shot below has non-default values for “Delay Start Time” and “Disable During Warmup”. The property description for “Delay Start Time” also describes how the “Disable During Warmup” setting affects when the Scenario actually starts:

NewScenarioPropertiesDelayStart

The description implies but does not explicitly state this: if “Disable During Warmup” is false, the Delay Start Time is measured from the very beginning of the load test including any warm-up period.

For a complete list of the new Web testing and Load Testing features in VSTS 2010 Beta 1 check out Ed Glas's list.