Load Testing Web Services with Unit Tests

This post is going to focus on load testing web services.  The web services will be called from unit tests which will then be added to a load test.  The example will also show how to use timers in a unit test which will then show up in the load test monitor. 

I will use 2 Web Services for this example.  The default HelloWorld service and another service called HelloUniverse.  A slight delay has been added to each web service call to make it easier to see the timing differences in the load test. Here is the code for my sample web services.  You can add this to a Web Service web site project to run through the rest of this example.

using System;

using System.Web;

using System.Web.Services;

using System.Web.Services.Protocols;

[WebService(Namespace = "https://tempuri.org/")]

[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]

public class HelloService : System.Web.Services.WebService

{

    public HelloService () {

       

    }

    [WebMethod]

    public string HelloWorld() {

        //Sleep for half a second

        System.Threading.Thread.Sleep(500);

        return "Hello World";

    }

    [WebMethod]

    public string HelloUniverse()

    {

        //sleep for one second

        System.Threading.Thread.Sleep(1000);

        return "Hello Universe";

    }

   

}

 

Now we will create a unit test from the web service.  Follow this link for more in depth information on creating unit tests: Unit Test Walkthrough

1. Create a test project

2. Right Click on the test project and select Add Web Reference…

3. Enter the URL to your web service in the Add Web Reference dialog.  In my example, I entered the URL for the HelloService Service and I gave the reference a name of Hello.

4. Click Add Reference to add the reference.

5. The next step is to create the unit test.  Right Click on the project and select Add New Item…

6. Select Unit Test in the Add New Item dialog.

7. Now it is time to create the unit test which will call the web service.  My unit test looks like the following:

 

using System;

using System.Text;

using System.Collections.Generic;

using Microsoft.VisualStudio.TestTools.UnitTesting;

//add using reference to the webservice

using TestProject1.Hello;

namespace TestProject1

{

    /// <summary>

  /// Unit tests for HelloService

    /// </summary>

    [TestClass]

    public class UnitTest1

    {

        public UnitTest1()

        {

           

        }

        [TestMethod]

        public void SayHello()

        {

            //Create the HelloService Object

            HelloService hello = new HelloService();

            //call hello world service

            string response = hello.HelloWorld();

            //verify it was successful

            Assert.Equals(response, "Hello World");

     //call hello universe service

            response = hello.HelloUniverse();

            //verify it was successful

            Assert.Equals(response, "Hello Universe");

        }

    }

}

 

8. The next step is to verify the unit test works as expected.  You can do this by running the unit test from the Test View window.  You can open the test view window from the Test Menu.  Test -> Other Windows -> Test View

If the test is working as expected, we can create the load test.  Follow this link for more information on Load Tests:  Load Tests

1. Right click on the test project, select Add and then select Load Test… in the fly out menu.

2. The previous step will launch the Load Test Wizard.  Let’s walk through the wizard:

a. Click Next on the first screen.

b. The second screen defines the load test scenario.  Enter a name for the scenario.  Think Time profile does not affect unit tests.  If you want to add a delay between iterations of a unit test, you can Change the Think Time between test iterations. 

c. The next pane is for selecting the load profile.  There are 2 profiles available to you: Constant and Step Load.  Select the profile and settings for your test.

d. The next pane is for Adding tests to the load test.  Click the Add button and add your unit test to the load test.

e. The next pane is for adding counter sets to the load test.  Please follow the above link for more information on counter sets.  I would suggest mapping the IIS and ASP.Net counter sets to your web server.  This will instruct the load test to collect the counters defined in these sets on the web server machine.  If you do map counter sets to the web server, then the user that runs the load test needs to have permission on the web server for collecting performance counters.

f. The next pane is for configuring some other load test settings such as the duration of the load test.  Set the duration to 5 minutes so we can smoke test the Load Test.  Click Finish when you have completed these settings.

 

After clicking finish, you will be brought into the Load Test Editor.  At this point, you can click the play button on the load test editor tool bar to run the load test.  Clicking play will launch the Load Test Monitor.  The default graph will have 2 lines: User Load and Tests/Sec and it will look like the following:

 

 

You may also want to add the specific counters for this test.  These are found in the counter tree under: Scenario1 -> SayHello ->. 

 

 

You should also look at the counters which you are collecting on the web servers and other machines under test to see they are handling the user load.  If any threshold violations or errors have occurred, a link will appear on the status bar.  This is a good spot to start looking for problems.

 

Next, You should look through the available tables.  The test and transaction tables will provide useful information on how long it is taking to execute the web service calls.  The unit test which I have defined above does not have any transactions defined, so the transaction table will be blank.  Here is the Test Table:

 

Transactions are useful if you want to collect timing information on a subset of an entire test.  For example, the test defined above makes 2 web service calls.  The test table and average test time counter give me information on how long it takes to make both calls, but what if I want information for each call?  One way to do this is by defining transactions, which are also called timers in unit tests.  Here is the same unit test, but with 3 timers defined.  One for both calls, one for the HelloWorld call and one for the HelloUniverse call.

using System;

using System.Text;

using System.Collections.Generic;

using Microsoft.VisualStudio.TestTools.UnitTesting;

//add using reference to the webservice

using TestProject1.Hello;

namespace TestProject1

{

    /// <summary>

    /// Unit tests for HelloService

    /// </summary>

    [TestClass]

    public class UnitTest1

    {

        TestContext testContextInstance1;

        public UnitTest1()

        {

        

        }

        //used to call the base methods of TestContext

        public TestContext TestContext

        {

            get { return testContextInstance1; }

            set { testContextInstance1 = value; }

        }

        [TestMethod]

        public void SayHello()

        {

            //Create the HelloService Object

            HelloService hello = new HelloService();

            //Start HelloBoth and HelloWorld timers

            TestContext.BeginTimer("HelloBoth");

            TestContext.BeginTimer("HelloWorld");

            //call hello world service

            string response = hello.HelloWorld();

            //end HelloWorld Timer

            TestContext.EndTimer("HelloWorld");

            //verify it was successful

    Assert.Equals(response, "Hello World");

            //Start HelloUniverse timer

            TestContext.BeginTimer("HelloUniverse");

            //call hello universe service

            response = hello.HelloUniverse();

            //end HelloUniverse Timer

            TestContext.EndTimer("HelloUniverse");

            //end HelloBoth Timer

            TestContext.EndTimer("HelloBoth");

            //verify it was successful

            Assert.Equals(response, "Hello Universe");

        }

    }

}

Save and compile the unit tests then run the load test again. This time when you go to the transactions table, you will see the 3 transactions. This shows you that the unit test spends more time in the HelloUniverse transaction than the HelloWorld transaction.

 

 

 

Another option that you have with timers is you can graph them. Go back to the graph view. The timer/transaction counters are stored under the test they are defined in. For my example it is: Scenario1->SayHello ->Transactions->HelloBoth->Avg.Transaction Time.

 

 

 

You also have the option to run multiple unit tests inside of a load test. This allows you to test multiple web service or other api calls from one test and to see how your servers react to having each of the services called together. When adding mutliple tests, you can set a percentage on each test which indicates how often the test will be called. For example, now suppose my unit test class looks like the following. Notice that 2 unit tests are defined:

 

using System;

using System.Text;

using System.Collections.Generic;

using Microsoft.VisualStudio.TestTools.UnitTesting;

//add using reference to the webservice

using TestProject1.Hello;

namespace TestProject1

{

    /// <summary>

    /// Unit tests for HelloService

    /// </summary>

    [TestClass]

  public class UnitTest2

    {

        TestContext testContextInstance1;

        public UnitTest2()

        {

        }

        //used to call the base methods of TestContext

        public TestContext TestContext

        {

            get { return testContextInstance1; }

            set { testContextInstance1 = value; }

        }

        [TestMethod]

        public void CallHelloWorld()

        {

            //Create the HelloService Object

            HelloService hello = new HelloService();

            //Start HelloWorld timers

            TestContext.BeginTimer("HelloWorld");

            //call HelloWorld service

            string response = hello.HelloWorld();

            //end HelloWorld Timer

            TestContext.EndTimer("HelloWorld");

            //verify it was successful

            Assert.Equals(response, "Hello World");

           

        }

        [TestMethod]

        public void CallHelloUniverse()

        {

            //Create the HelloService Object

     HelloService hello = new HelloService();

            //Start HelloUniverse timer

            TestContext.BeginTimer("HelloUniverse");

            //call HelloUniverse service

            string response = hello.HelloUniverse();

            //end HelloUniverse Timer

            TestContext.EndTimer("HelloUniverse");

            //verify it was successful

            Assert.Equals(response, "Hello Universe");

        }

    }

}

I can now add both the CallHelloUniverse test and the CallHelloWorld test to a load test and both will be executed during the load test execution. If I know that about 75% of the users hit the CallHelloWorld method and 25% hit the CallHelloUniverse service, I can set these as the percentages in the load test and this is about how often each test will be called. If you perform these steps and run the load test, you can easily see the percentage differences by looking at the test table in the Load Test Monitor.

 

 

I hope this gives you an idea on how you can use unit tests to call web services and hit these services under load.   Please let me know if you have more questions on how this process works.

 

Here are the same code examples in VB:

Web Service:

Imports System.Web

Imports System.Web.Services

Imports System.Web.Services.Protocols

<WebService(Namespace:="https://tempuri.org/")> _

<WebServiceBinding(ConformsTo:=WsiProfiles.BasicProfile1_1)> _

<Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()> _

Public Class HelloService

    Inherits System.Web.Services.WebService

    <WebMethod()> _

    Public Function HelloWorld() As String

        System.Threading.Thread.Sleep(500)

        Return "Hello World"

    End Function

    <WebMethod()> _

    Public Function HelloUniverse() As String

        System.Threading.Thread.Sleep(1000)

        Return "Hello Universe"

    End Function

End Class

Unit Test:

Imports System

Imports System.Text

Imports System.Collections.Generic

Imports Microsoft.VisualStudio.TestTools.UnitTesting

Imports TestProject1.Hello

<TestClass()> Public Class UnitTest1

    <TestMethod()> Public Sub SayHello()

        'Create the HelloService Object

        Dim hello As HelloService = New HelloService()

        'call hello world service

        Dim response As String = hello.HelloWorld()

        'verify it was successful

        Assert.Equals(response, "Hello World")

        'call hello universe service

        response = hello.HelloUniverse()

        'verify it was successful

        Assert.Equals(response, "Hello Universe")

    End Sub

End Class

Unit Test With Transcations:

Imports System

Imports System.Text

Imports System.Collections.Generic

Imports Microsoft.VisualStudio.TestTools.UnitTesting

Imports TestProject1.Hello

<TestClass()> Public Class UnitTest1

    Dim testContextInstance1 As TestContext

    'used to call the base methods of TestContext

  Public Property TestContext() As TestContext

        Get

            Return testContextInstance1

        End Get

        Set(ByVal Value As TestContext)

            testContextInstance1 = value

        End Set

    End Property

    <TestMethod()> Public Sub SayHello()

        'Create the HelloService Object

        Dim hello As HelloService = New HelloService()

        'Start HelloBoth and HelloWorld timers

        TestContext.BeginTimer("HelloBoth")

        TestContext.BeginTimer("HelloWorld")

       'call hello world service

        Dim response As String = hello.HelloWorld()

        'end HelloWorld Timer

        TestContext.EndTimer("HelloWorld")

        'verify it was successful

        Assert.Equals(response, "Hello World")

        'Start HelloUniverse timer

        TestContext.BeginTimer("HelloUniverse")

        'call hello universe service

        response = hello.HelloUniverse()

        'end HelloUniverse Timer

        TestContext.EndTimer("HelloUniverse")

        'end HelloBoth Timer

     TestContext.EndTimer("HelloBoth")

        'verify it was successful

        Assert.Equals(response, "Hello Universe")

    End Sub

End Class

Unit Test class with 2 unit tests:

Imports System

Imports System.Text

Imports System.Collections.Generic

Imports Microsoft.VisualStudio.TestTools.UnitTesting

Imports TestProject1.Hello

<TestClass()> Public Class UnitTest2

    Dim testContextInstance1 As TestContext

    'used to call the base methods of TestContext

    Public Property TestContext() As TestContext

        Get

            Return testContextInstance1

        End Get

        Set(ByVal Value As TestContext)

            testContextInstance1 = value

        End Set

    End Property

    <TestMethod()> Public Sub CallHelloWorld()

        'Create the HelloService Object

        Dim hello As HelloService = New HelloService()

        'Start HelloWorld timers

        TestContext.BeginTimer("HelloWorld")

        'call HelloWorld service

        Dim response As String = hello.HelloWorld()

     'end HelloWorld Timer

        TestContext.EndTimer("HelloWorld")

        'verify it was successful

        Assert.Equals(response, "Hello World")

    End Sub

    <TestMethod()> Public Sub CallHelloUniverse()

        'Create the HelloService Object

  Dim hello As HelloService = New HelloService()

        'Start HelloUniverse timer

        TestContext.BeginTimer("HelloUniverse")

        'call HelloUniverse service

        Dim response As String = hello.HelloUniverse()

        'end HelloUniverse Timer

        TestContext.EndTimer("HelloUniverse")

        'verify it was successful

        Assert.Equals(response, "Hello Universe")

    End Sub

End Class