Designing for Performance

  • Article

I wrote this article back in July and it ended up being the basis of this video (scroll to where it says “Thinking about Performance” and choose a speed)

I was going to have the article edited and published seperately but somehow that never happened, so here it is now... the content isn't terribly new but it's kinda handy to have some of it in written form.

Designing for Performance

I’m a “performance guy”. That means I care deeply about how fast things are, and about keeping them small and tight. I think I’m a lucky performance guy because I actually get paid to do it – I work on the .NET Runtime. Some days being a performance guy isn’t much fun. Today, I have some things I want to get off my chest.

I think this article is a reaction to hearing the phrase “Premature Optimization is the root of all evil” one time too many. So I’m here to tell you The Truth. I’ve done a long and careful study of the Roots of Evil and my conclusion is that Premature Optimization isn’t the root of it. It turns out that the Actual No-Kidding-Around Root-Of-All-Evil is Sloppy Engineering.

Now, please don’t get me wrong. I’m not here to tell you that premature optimization is good for you and so you should get at least five milligrams of it daily. In fact, with the possible exception of “lottery winnings,” it’s hard to think of any words I could put after “premature” to yield something that seems like a good thing – it’s a pretty negative way to start a phrase. So I think it’s very fair to say that you can get yourself into a lot of trouble if you start optimizing your code too soon. But Sloppy Engineering is worse.

You see, the danger of that “Premature Optimization…” phrase is that it encourages good engineers to not think about performance until far too late in their development cycle and they have a really swell sounding reason to not do it.

It’s usually pretty obvious when a team has fallen into this trap – they’re close to releasing their software, everything is “working”, but they have abysmal startup time, or they’re using far too much memory, or something else of that ilk. Now they’re desperate for advice. If you’re very lucky performance problems can be fixed after the fact. But, as often as not, it will take a great deal of effort to get your code to where it needs to be for acceptable performance. This is a very bad trap to fall into. At its worst, you’ll be faced with another memorable, and sometimes job-ending, quote:

“This is never going to work. You’re going to have to start all over.”

If things have gone really badly then a team with a poor performing solution is going to have to do some major rewriting, even complete rewriting. This happens if the design that was chosen to address their problem fundamentally requires the use of a basic service or technique that is far too costly in terms of space or speed for the scenario their project was designed to work in. I’d like to say that this hardly ever happens but I’ve seen two examples just this week.

On the one hand you have all these wonderful services that are so very easy to use, we’ve gone to great pains to make them as easy to use as possible after all, but they sometimes come at significant cost and aren’t always appropriate for every problem. I’d say the most frequently misused classes are from the System.Xml and System.Reflection namespaces.

You really do not want performance disasters to happen to you.

Honest.

So how do you prevent this sort of thing? Planning is everything. Good performance is a key design consideration of any serious project and its relative importance should be carefully weighed with all your other major considerations. So while I think it’s a mistake to consider micro-optimizations early in your project’s life, I think it is vital to consider the performance goals of your project and design a solution that is going to be able to meet those goals from the outset. If you are not engineering a solution that is highly likely to meet your project’s goals – all the goals – then that’s Sloppy Engineering and that really is The Root of All Evil.

So what to do? Well this article is about performance so I want to give a little advice about how to deal with the performance related goals of your project but some of my advice is equally applicable to other aspects of your project’s needs for success.

The first thing you have to do is know what success looks like. Sadly there is not one universal set of success metrics that works for every project. You’ll have to think about what’s important to your project – it might be startup time, memory usage, disk usage, computation speed (the frame rate of your game) or a mix of all these things. It’s very likely that you’ll need to do some up front research to get this level of understanding and the best place to get it is from your customers. Once you know what success looks like, then you will be in an excellent position to make budgeting decisions.

I’m a very big fan of performance budgets. I’m always asking to see the budgets because they tell me a great deal about the relative importance of the work going on in a project and they help me to quickly identify the big challenges. Whether I will be working on the project myself, or simply reviewing it at some stage, the performance budgets give me something concrete to focus on.

But I think I’m ahead of myself. I haven’t told you what I mean by performance budgets. They’re nothing more than a statement of the maximum cost that a particular feature or unit in your project can afford to pay against each of the key performance goals. For example, if your performance goals call for your application to be initialized in less than ten seconds then you might budget a maximum of one second to each of your ten modules. Or, if your application is a server and it must do 3000 transactions per second per CPU then the total time per transaction cannot exceed 333 microseconds.  

Three hundred and thirty-three microseconds is a very liberating number if you’re an engineer. That number is going to help you a lot. Maybe your boss wanted “HyperTrans” in his server but, knowing that your budget is 333 microseconds, you can now confidently stare him down and say “Yo boss, everyone knows that a HyperTran takes 300 microseconds… that leaves only 33 microseconds to do the other useful work and we all know that isn’t happening…” Obviously the HyperTran approach is right out.

In fact, dozens of other approaches are probably also right out as well. That budget has provided you with great guidance as to what you can reasonably do. You can quickly eliminate a variety of options and focus on the winners. Now, don’t go the other way and micro-tune the winners, but do conduct a suitable number of experiments up front to verify your choices. Do your homework.

At this point I’m often asked “How much up front work should I do?” and sadly I don’t have a terribly useful answer for you. I usually answer the question with a glib phrase like “You should do exactly as much as is called for and not one shred more.”

Well in the interest of actually trying to be helpful I do have some advice. I look at this from a risk-management perspective. If we’re talking about only a few hours worth of work to build the new code then you probably shouldn’t spend a week doing an investigation. On the other hand if the project will require several man-years to complete I dare say that more than 5 minutes of up-front planning is warranted.   The more work is at stake the greater the need to reduce the risk of failure down the road. The planning should put you in a situation where you are taking the proper amount of risk.

With the perfect amount of planning done, the budget still is going to keep working for you. While development goes along you can check your work against your goals and create tests to make sure the work is on track and stays there. You and your testing organization are well armed to find performance issues because they’ll know what to measure. Measure your work early and often, but don’t just measure, pay attention and adjust your plans.

If things went great then as you are going along you’ll be able to periodically (don’t overdo this) verify your progress and reassure yourself that things are converging as you intended them to. If things start going badly, you mustn’t just plod along like a mindless zombie. Take advantage of the planning you did to see where and why things are not going as they should. Which aspects are more costly than anticipated? Is it a design problem or is it just tuning that’s needed? If it’s just tuning perhaps there isn’t a problem at all, but if it’s more you may have to go redesign or it may be time to cut your losses. Whatever you do, don’t just blindly finish on a course that is doomed to fall short of the projects requirements, that helps no-one. The worst kind of Sloppy Engineer is one that inflicts his/her work on the world at large.

It may sometimes be difficult to remember to take the planning steps. In the software business we’re often under tremendous time-pressure. I can only emphasize that the greater the pressure of the situation the more important it is that we do good engineering because we have less time to fix things up later. Ultimately good planning saves you time.

In summary:

  • Think carefully about the goals of your project,

    • Include performance goals among them

  • Use those goals to create budgets for each project area

  • Design up front in such a way that you have a good expectation of meeting those goals

    • Do early homework researching the cost of services you intend to use to form the basis of those conclusions
    • Build prototypes to verify costs if necessary
    • Do not over-research, use the planning phase to manage project risk to the right level for your project

  • Measure your progress against those goals regularly and make corrections as needed

    • Build in some time for course corrections into your schedule
  • Do not be afraid to cancel things that are clearly not going to work – better to cut your losses to give you time to build something that will work
  • Do not confuse a design that duly considers performance up-front with “premature optimization”
  • Do not “write it the easiest way first and make it fast later” because sometimes the easiest way simply can’t be made to go fast

And most important of all:

  • Do not blindly ignore these rules and expect to get good performance in your projects automatically
  • Do ignore each and every one of my suggestions when you have a thoughtful reason to do so

Thanks for listening.