Which comes first, the Process or the Tool?

Prologue I was just going through my email and came across a question on one of our discussion aliases that asked if it was possible to deliver an ALM assessment for a customer without worrying about the tool set they currently use. The customer wanted to have the assessment focus on their process. The person…

0

Dissecting the Visual Studio Load Test Results Database (Part 6–Data Warehousing)

In the previous post, I showed how I use a couple of stored procedures to recalculate data based on various criteria. In this post, I am going to share a few data tables that I use to store “Warehousing” data to make reports faster and easier to create. In the next post I will show…

0

Dissecting the Visual Studio Load Test Results Database (Part 4–Building Better Views)

In the previous posts of this series, I focused on showing you the built in features of the Load Test Results Database. The next few posts will focus on customizing and extending the database. The Mystery of the Perfmon Page Counters There are two ways you can look at the details for pages, transactions, etc…

0

Dissecting the Visual Studio Load Test Results Database (Part 3–Excel Reporting)

In Part 3, we look at the sprocs and calls used to build reports in Excel. The Excel Reporting Feature In the 2010 release of Visual Studio Ultimate, a new feature was added that allows you to use Microsoft Excel to create either a Run Comparison or Trend Analysis report. You can read the standard…

1

Dissecting the Visual Studio Load Test Results Database (Part 1–Intro to the Schema)

Sean Lumley wrote a good blog post years ago about the actual schema for the Visual Studio Load Test results database, but even with that post, navigating the DB and figuring out how things tie together can be confusing at the least, and sometimes almost impossible to figure out. I have been working inside the…

1

How Does the User Pacing model differ from the Number of Tests Started model

In my post on User Load Profiles, I talked about choosing one of two different load models, based on a couple of the options in Visual Studio. However, I have still received a lot of requests to clarify these two models and to show how I can apply one of the models to the testing…

0

All About Load Test Results and the Results DB (Part 5–Requests vs. Pages vs. Transactions

I was recently asked about the details of transaction timers and realized that I have not made a post that clearly shows the differences between the various timers that Visual Studio collects and reports on. While there is a good help file page on this in the online documentation (https://msdn.microsoft.com/en-us/library/ms404656.aspx), I thought I would show…

0

Test And Attack – The Art of Testing WITH…

My old team coined a phrase that they use to describe their Performance engagements called “Test and Attack.” It is meant to conjure up a picture that can represent exactly what happens during the execution phase of a performance tuning and optimization engagement. So what does happen? Test With You, Not For You We refer…

0

Should I Setup My Rig Outside of My Environment’s DMZ?

My old team would get this question all of the time when we were scoping potential customer engagements. The customer wanted to make sure we could simulate the “Internet” for their testing. We have hardware WAN simulators in our labs, so we could easily do this. However, when we were asked to include them in…

0

[FIXED] Importing and Exporting Perfmon Counters [download]

(NOTE: A bug was reported about errors (missing stored procedures) when trying to add the SQL file to the DB.I have fixed that bug and moved the downloads to CodePlex. Please use this link to get the tool: https://perfmonimportexport.codeplex.com/ ) I am posting this utility to help people who need to import or export perfmon…

3

LTP – Real World Example: Calculating Work Load From Historical Data

Recently, I saw a question on an internal alias that spawned a very useful dialog I wish to share below. This example shows how I approach calculating the work load and load pattern when using a mix of historical data and the need to project future data. Below is the relevant part of the email…

0

Code Snippets, Templates and Perfmon Countersets [download]

I am sharing a set of tools I use during many of my engagements and projects. These tools have been written by myself and my ITSM teammates over the past several years.  The zip file includes the following (each item has a Word document explaining how to install and use): C# Project and Item Templates…

0

OFF TOPIC: Introducing the Premier Developer Blog Site

I am adding this post to showcase the work of my teammates here at Microsoft. The below information is from the official announcement. I am very honored to be highlighted in their initial rollout. Please be sure to link to this new site, as I am sure it will be teeming with great information over…

0

All About Load Test Planning (Part 5-Load Profile Additional Considerations)

In the previous post, I showed you how to come up with the profiles to use in a test as well as the numbers to plug into the profile. I also showed two fairly simple examples of load profiles that you might generate. In this post, I will show you some more examples of profiles,…

1

All About Load Test Planning (Part 4-Building the Load Profile)

Let’s start off with a review of this series. In the first post, we set our goals and objectives and defined the “purpose” of our testing. In the next post, we looked at data considerations that are independent of our goals, but still need to be considered. In the third post, we laid out the…

1

All About Load Test Planning (Part 3-Proper Use Cases and Scenarios)

PART 3 in a series of posts on Load Test Planning As of this point, we should have well defined Goals, Objectives and success criteria (part 1) and we should have an idea of how we want to consider building our data (part 2). Now we want to lay out the basic scenarios and use…

5

All About Load Test Planning (Part 2-Load Patterns and Data Considerations)

PART 2 in a series of posts on Load Test Planning In my previous post, I talked about the importance of goals, objectives and success criteria. Now that we have that information together, we can consider the next part, which is going to be defining the style of test that I want to use (NOTE:…

2

All About Load Test Planning (Part 1–Introduction)

PART 1 in a series of posts on Load Test Planning Introduction As part of my work inside Microsoft, I have to interface with several different groups of people, from sales to account management to the technical delivery people, and also to different product group teams. I have learned some very valuable lessons from these…

1

Off Topic – Using Outlook to Manage Tasks

I struggle a lot with trying to manage my daily chores and tasks, especially since I suffer from ADHD (clinically diagnosed). I recently asked my teammates how they managed their list of tasks and action items and ToDo lists, and I was surprised to hear that many of them are also looking for a simple…

0

[UPDATED] Adding Timers to Coded UI tests running in Load Tests

  I have uploaded a sample App and Coded UI test project for the app with the below code in it. This sample also shows a couple of examples of how to wait for events and how to randomly pick values from screen items. Enjoy and please send feedback. By default, Visual Studio does not…

4

(UPDATED) How can I create thousands of clients to execute an End To End load test?

<After initially posting this article, I got some feedback from a customer who said that his company’s apps aren’t using web protocols. Their apps use RDP. I realized that I was making an assumption, and I can’t afford to do that in my job, so I am updating the article to cover "all" protocols> With…

0

"Extrapolation, Educated Guesses and Dart Boards, Oh My"

Here is another common question in the testing world: “What is the best way to take the results from our tests with a small load and estimate how the system will behave under a full load?” There are many different reasons for asking this question, not the least of which is because testers often do…

0

To “Think” or “Not to Think”

This is one of my favorite topics to debate with people, and it also seems to be one that stirs up the most passion amongst testers who have an opinion. However, as with so much information in testing strategies, there is NOT ONE CORRECT ANSWER. What I hear all of the time… “Driving load without…

0