How to get 90th and 95th percentile information as well as timing details for every page, test and transcation
This blog post will show you how to collect timing information for every single page, test and transaction as well as 90th and 95th percentile data. When a load test runs it is collecting a sample of each performance counter at some configured interval. By default this interval is every 5 seconds. So every five seconds, the load test engine will attempt to collect information about each of the performance counters that are part of the load test. These samples can include information about multiple data points. For example, if one page executes 5 times during the 5 second sample, you still get just one data point for the sample.
This level of detail is sufficient in many cases, but there might be times that you want to collect information about every single page request or every single transaction in your load test. We provide the ability to do just this. We provide a nice UI for viewing the performance counter data, but we do not however provide a UI for viewing this detail data. So you would need to pull the data into excel or using some other reporting package. One of the sample reports I put together does display this detailed data for pages: Reports
This data is collected by setting the Timing Details Storage property on the run settings node of your load test. There are 2 choices for this property.
1. All individual details – This option does 2 things. It will collect and store information about every single page, test and transaction in your load test repository. It will also generate the 90th and 95th percentile information for the pages, tests and transactions. Depending on the length of your load test and the number of pages, tests and transactions, this could a lot of space in your database.
2. Statistics Only – This option will collect the information for every single page, test and transaction. It will then use the information to calculate the 90th and 95th percentile information and then it will discard the data. This option is good if you want the percentile information but do not want to report on all of the individual details.
Here is how you would go about collecting the data:
1. Open your load test in the load test editor
2. Click on the run setting node
3. Bring up the property browser and locate the Timing Details Storage property
4. Set this to either “All individual details” or “Statistics Only”
5. Save and run your test
6. When the test is complete, transition to the post run UI. Now go to the tables for pages, tests and transactions. You will see data for the 95th percentile. If the data for 90th percentile is not showing, right click on the table, select add/remove columns, then choose 90th percentile.
If you store all the details and want to pull it into a report, you will find the data in the following tables: LoadTestTestDetail, LoadTestPageDetail, and LoadTestTransactionDetail. See Bill Barnett’s post on the load test schema for further information about these tables: Load Test Repository
I hope this helps explain how to collect more information while running your load tests.