Troubleshooting the VSTS Load Testing IP Switching Feature

  • Article

First some background on how IP Switching works with VSTS, which may differ from other load test tools you have used:

Many other load testing tools that support IP Switching (also known as IP spoofing) require you to add the IP addresses to the network interface card (NIC) manually, and then the IP addresses are configured on the NIC until they are manually deconfigured.
 
VSTS on the other hand, dynamically adds the IP addresses to the chosen NIC on the agent(s) when the test run starts, and removes them when the test run stops.

If you are having trouble getting IP switching to work correctly, I recommend the following setup and troubleshooting steps:

1) Make sure that the Agent Service is running as a user than is an admin on the agent machine; this is required because the agent service attempts to configure the IP addresses specified in the agent properties on the chosen NIC, and admin permission is required to do this.

2) Make sure that none of the IP addresses in the range specified for a particular agent are already configured on the chosen NIC.

3) Enable verbose logging for the Agent Service:

  * Edit the file QTAgentService.exe.config:
        For VSTS 2005: <Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTAgentService.exe.config
        For VSTS 2008: <Program Files>\Microsoft Visual Studio 9.0 Team Test Load Agent\LoadTest\QTAgentService.exe.config

Change:
     <add key="CreateTraceListener" value="no"/>
to
     <add key="CreateTraceListener" value="yes"/>

Change:
      <add name="EqtTraceLevel" value="3" />
to:
      <add name="EqtTraceLevel" value="4" />

  * Restart the Load Test Agent service

  * The file "VSTTAgent.log" will be created in the same directory as QTAgentService.exe.config.

  * Re-run the load test with verbose logging configured, and look for lines in the log file that contain the text:

     "Attempting to configure IP address:"
     "Configured IP address:"

    This will tell you whether or not you the agent service is attempting to configure the IP address you've specified.
    If you see the "Configured IP address:" line, it has succeeded in configuring this IP address.  If not, there should be
    some error logged.  

    If you have verified the items in step 1 & 2 above, and the log indicates that the configuration of the IP address is failing
    but you cannot determine the
    cause of the failure from the error message in the log (or if there is no error message in the log), post a new thread to the
    Web and Load testing forum, or open a Microsoft Support incident for further assistance, and provide details on the setup
    including the relevant portions of the log file.

4) Make sure that the load test you are running is set to use IP Switching: Click on each of the "Scenario" nodes in the load test editor,
go to the property sheet, and verify that the "IP Switching" property is set to True (normally it should be since this is the default, but it's worth checking).

5) Enable verbose logging for the Agent process.

If the log file created in step 3 shows that the IP addresses are being successfully configured, the next step is to check the agent process
log file to verify that the load test is actually sending requests using those IP addresses.

To enable verbose logging for the agent process:

  * Edit the file QTAgent.exe.config:
        For VSTS 2005: <Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTAgent.exe.config
        For VSTS 2008: <Program Files>\Microsoft Visual Studio 9.0 Team Test Load Agent\LoadTest\QTAgent.exe.config

Change:
     <add key="CreateTraceListener" value="no"/>
to
     <add key="CreateTraceListener" value="yes"/>

Change:
      <add name="EqtTraceLevel" value="3" />
to:
      <add name="EqtTraceLevel" value="4" />

  * The file "VSTTAgentProcess.log" will be created in the same directory as QTAgent.exe.config.

  * Re-run the load test, and look for lines in the log file that look something like:
        "Bound request on connection group M to IP address NNN.NNN.NNN.NNN"

If verbose logging is enabled and these lines are present in the log file, IP Switching should be working.

6) If the number of unique IP addresses being used as shown by the log entries in step 5 is less than the number in the range that
was configured, it could be because your load test is configured to use a connection pool with a smaller number of connections than
the number of IP addresses specified.   If this is the case, you can increase the size of the connection pool, or switch to "Connection per User"
mode in the load test's run settings properties.