Visual Studio Team Services - Creating a build pipeline (Part 4)

In part 1 a basic Azure Web App website was built and deployed using VSTS Build and Part 2 illustrated how additional files could be included when building a web project.  Part 3 demonstrated how Gulp could be added to the VSTS Build  pipeline.  This post is an introduction to bringing in Load Testing to the build pipeline. 

Performance Tests

In the Azure Portal there is the ability to run light-weight performance test by performing a GET request against a specified page.  For details see the references section, but the following is a quick summary.  First, in the Development Tools -> Performance tests section use New to create a new test:

2016-08-19_0916

The Manual Test allows for a test allows for a number of users over a period of time to request a page from the web app.

2016-08-19_0918

The performance test does maintain a history of recent tests allowing for changes in behavior over time to be identified.

2016-08-19_0934

As well as being able to identify failures and to get an indication of the general performance of the site.

2016-08-19_0934_001

References

Web Performance and Load Test Project

For more advanced testing scenarios, Apache JMeter or Microsoft's Web Performance Tests can be used.  In this post a load test will be created using Visual Studio so the Web Performance Test will be used.

Note: Web Performance and Load Test Project is a Visual Studio Enterprise feature. Try Apache JMeter for professional and community editions.

To begin a new Web Performance and Load Test Project is added to the solution:

2016-08-19_0822

Add Recording

The easiest way to get started is to use the Add Recording feature.  By adding a recording, a scenario can be quickly added to the web test and then edited if required.

2016-08-19_0837

If this is the first time, it is likely two add-ons will need to be enabled:

2016-08-19_0839

Make sure the two add-ons below are Enabled.

2016-08-19_0839_001

After performing some steps within the web app, the recording can be stopped.  In some cases and especially on development machines, additional activity might be captured and added to the web test.  These can be simply deleted from the steps as illustrated below:

2016-08-19_0918_001

Tip - If you are developing for multiple deployments or environments (re., dev, test, prod), it is useful to parameterize the web service calls.

2016-08-19_0942

References

Add Validation Rule

Once a test plan is created, additional steps and rules can be applied.  For example, if we wanted to add a rule to validate if the text "Enchilada" was returned in a request, then we could add a rule by right clicking on the web request and selecting Add Validation Rule:

2016-08-19_0944_001

The Find Text rule can be added to simply check for a specified text in the response:

2016-08-19_0945

As the web app does not return enchilada on the dashboard, the validation rule fails as shown below and provides detail as to reason for the failure:

2016-08-19_0944

Using the Application Insights monitoring that was create in Part 2, we can see the contact page appears to be slower page compared to the other pages:

2016-08-19_0950

The following demonstrates how to create a validation rule for response time.

2016-08-22_1536

If the response time is greater than 50 milliseconds then the request is considered a failure:

2016-08-19_0958_001

Tip - The sequence and behavior of the test steps can be altered.  For example, if the test should stop on a failure, this can be specified on the test plan properties:

2016-08-19_0954

Below shows the test run being aborted after the contact web request failure:

2016-08-19_0958

References

Load Testing

Once one or more web tests have been created, it is very easy to enable Load Testing.  In the same test project, add a load test to start New Load Test Wizard:

2016-08-19_1002

For this example, a cloud-based load test will be created:

2016-08-19_1003

The default datacenter, in this case the same as where the web test is hosted, will be chosen:

2016-08-19_1003_001

The duration of the test will be five minutes:

2016-08-19_1004

The scenario name as well as think times can be specified next:

2016-08-19_1004_001

As an illustration of step load, the user count will start at 2 users and then gradually increase to 50 by adding 2 every 10 seconds:

2016-08-19_1005

The test mix can also be specified when running multiple tests in one simulation:

2016-08-19_1006

In this scenario, the single web test will be added:

2016-08-19_1006_001

Note the ability to alter the distribution when multiple tests are specified:

2016-08-19_1006_002

Another nice feature is the ability to distribute the load between different browser types:

2016-08-19_1007

After the load test wizard has been finished, the load load can be launched!

2016-08-19_1009

Once the load test has completed, a report is generated showing the results.  Note the how as the user count increases (User Load), the Average Response and Average Page Time does not increase.

2016-08-19_1020

References

Setting up Load Testing in the build pipeline

Setting up a Load Test task is surprisingly easy.  The first step is to add a Cloud-based Load Test task to the build pipeline:

2016-08-23_0803

The task can be used to launch the load test on a registered connection allowing for the load test to be run on on-premises or cloud based servers.  The first step is to register a new connection.  Note the short cut:

2016-08-23_0803_001

The connection type is a Generic Service Endpoint.

2016-08-23_0804

In this example, the connection will be for the VSTS account that is running the load tests.

2016-08-23_0810

The remainder of the settings are relatively straightforward and reference the load test and settings that were created in the steps above.  One thing to note is the helpful select dialog available:

2016-08-23_0813

Once the task has been configured then the load test will be run as part of the build:

2016-08-23_0919

References

Summary

The following shows the current build pipeline:

2016-08-23_0921

Next Steps

In the next post, slots will be introduced to our build process and how they can work with our Load Test.