How to simulate IE Caching in VSTS 2008

Sometimes it’s beneficial to run load test simulating IE cache. For example, if application is used frequently by repeated users, or if project team wants to set up production monitoring system that measures user experience with cache.

There are two places in VSTS where cache should be enabled to achieve close IE cache simulation:

1) In the web test, “Cache Control” property should be set to True for every single request (see Note below for the code example):

2) In the load test properties, “Percentage of New Users” property should be set to 0:

Note: Setting cache control property to true for every single web request manually would be feasible if web test had only a few requests. But when the amount of requests grows to hundreds or more, it’s beneficial to use the code below, kindly provided to us by Sean Lumley, Senior SDE from VSTS team. You would have to convert your web test to coded C# test to use this code:

public class WebTest1Coded : WebTest

{

public WebTest1Coded()

{

this.PreAuthenticate = true;

this.PreRequest += newEventHandler<PreRequestEventArgs>(WebTest1Coded_PreRequest);

}

void WebTest1Coded_PreRequest(object sender, PreRequestEventArgs e)

{

e.Request.Cache = true;

}

        public override IEnumerator<WebTestRequest> GetRequestEnumerator()

        {

            WebTestRequest request1 = new WebTestRequest("https://localhost/");

            request1.Encoding = System.Text.Encoding.GetEncoding("utf-8");

            yield return request1;

            request1 = null;

            WebTestRequest request2 = new WebTestRequest("https://localhost/");

            request2.Encoding = System.Text.Encoding.GetEncoding("utf-8");

            yield return request2;

            request2 = null;

        }

 

Parts of the code, which are in bold, are the only parts you would have to add to your coded web test. After applying this code, all your requests will have Cache Control property set to True.

Here is the table with results from 5 short load test runs, where we modified Cache Control property in the web test and/or Percentage of New Users in load test property:

 

1st run

2nd run

3d run

4th run

5th run

Cache control property

False

False

True (manual)

True (manual)

True (Coded)

 % of new users

100%

0%

100%

0%

0%

tests

12

12

12

12

12

total requests

2400

1918

1660

1177

1181

requests/sec

27

23.3

24.9

20.2

19.8

cached requests

1560

2231

2532

3225

3225

duration

88 sec

82 sec

66 sec

58 sec

59 sec

As you can see from the table above, combining those two cache settings gives us the most caching (cached requests in the table). Also, code example provided above, generates the same amount of caching as manual process would do. (Compare run#4 and run#5 results).

Another observation we noticed was related to actual cached user page load times. Enabling both caching properties in VSTS provided transaction times, which were very close to actual end user cached experience (no more than 1 sec difference on 5-10 sec transactions). Those numbers were close for our application, but this might not be true for every application. That’s why you should verify close matching of those numbers for your particular application.

---------------------------------------- 

Vitaliy Konev

Performance Engineer

Microsoft – ACE Team