Well our Patterns and Practices group has done it again with an excellent new guide, JD just announced the final version of Performance Testing Guidance for Web Applications.
As with the last guide I was asked to provide feedback along the way and so I was happy to write a foreword for them which I include here for interest. It's recommended reading 🙂
Foreword By Rico Mariani
It’s hard to imagine anything than is considered a more arcane art than performance tuning – unless perhaps it is performance testing.
If you were to go door to door between groups just within Microsoft you would find many different approaches with various different degrees of quality or success. Pretty much everyone will vow that their approach is certainly the one that is best for them – except maybe an honest few, who might say something more modest. Some have good reason to be confident because they really have studied the space very well. In my own experience at least, the situation is not altogether different outside of Microsoft than it is inside where I do my work. It’s a mixed bag, on a good day.
If I had to describe the most common problem I see in this space with one word it would imbalance. There are many aspects to testing and teams tend to unduly focus on one or another and then sometimes get blindsided by the ones they missed. Perhaps they’re only thinking about throughput – what about consumption? Perhaps only latency – what about smooth delivery? Perhaps only cost -- what about scalability?
You get great performance by balancing the key factors, considering them in your designs and then tracking them carefully. So perhaps the greatest service that a book like Performance Testing Guidance for Web Applications can provide to you is a broader understanding of what all the factors might be so that you have an excellent menu of considerations to choose from in your testing plan. Luckily, that is just what you’re going to get.
The Guidance that follows provides a great survey of the most important considerations: From how to understand and quantify your desired end user experience, how to choose key resources for study, to advice on summarizing results in a statistically meaningful way, and how to fit these practices into different software lifecycles. And even though the focus is squarely on web applications, the teachings are actually much more general and can easily be applied for many different kinds of applications.
Great engineering comes from creating predictable results at predictable costs. In fact, I like to say that if you’re not measuring you’re not engineering. This volume will provide you with the performance testing fundamentals to give you the ongoing metrics you need to do great engineering.