Learning as an engineer -- rising from the ashes of failed projects

Recently a senior program manager asked me for some reading suggestions and I wanted to share a book that had a big impact on how I view projects that I have worked on. The book is called "To Engineer is Human: The Role of Failure in Successful Design" by Henry Petroski (I have no connection to the author and never met him). This is one of my favorite books to give to engineers for holiday presents and I think I’ve probably given a copy to most of the senior managers on the Office team.

The book is a wonderful history of some engineering projects. The primary thesis that is well supported by the evidence is that for every successful engineering project there is a series of failures that preceded it. Mr. Petroski is a wonderful writer and the book is an easy and fun read. Of course one attraction is that the book talks about the failure of the Tacoma Narrows bridge, which is right by Seattle and an amazing site both in person and when you look at the famous pictures of its collapse. The book details other even more tragic events such as the Kansas City Hyatt walkway collapse. It shows how small design mistakes can be overlooked and also how future designs benefit from the failures. Often people who are not engineers really don’t quite understand how you learn and improve as an engineer by making mistakes, and obviously the challenge is that if your failures cause damage it is really unfortunate.  

I think a lot about this relative to the security issues we face in computer software today—we did not design for these and while we are doing everything possible to update old software the real benefit is coming from the new generations of software that were designed from the beginning to work in this environment (for example, in the 2 years that Office 2003 has been out we have only had 2 critical updates). I don’t want to make any excuses at all so don’t misinterpret that, but the learning that has taken place in our industry over the past 5 years has been immense and necessary for us to build products like Office 2003.

When I think about software projects that I have worked on from an OS in CS 415 at Cornell through an object database in graduate school all through Microsoft projects, the notion that failure often precedes success is one that is near and dear to me. Our industry is of course filled with failures that went on to become the basis upon which much learning took place (like the Newton to the Palm Pilot, for example, or the new TabletPC learning from the old PenWindows).

The reason this book resonates with me is because my first project at Microsoft was a pretty colossal failure. I was hired into a very small group (in business school lingo, the group would have been called a “tiger team” – a hand-picked group of experts with a specific short term mission). I don’t know how I ended up in this group since it was filled with incredibly senior developers and I was really intimidated. In any event, the mission we were given just before I started at MS was “build an application framework for Windows”. It sure seemed easy enough—we had a lot of great folks and even little me had just worked on Smalltalk frameworks for 2 years so we collectively thought “no problem”.

We worked for about a year and had a rather complete framework. We were pretty confident in our ability to use it to build robust applications in short order. So we decided to spend a month (a whole month!) building applications. Microsoft Money was under development so I figured I could spend my month building Money, which seemed simple enough. The framework had graphics, persistent objects, a UI model-view-controller framework, etc. ¡No problema!

After a month I had a fantastic application – it was able to successful draw a check on the screen. Needless to say our framework was too big, too slow, too complex for anyone to really use. It was a dismal failure despite all the right ingredients and doing all the “right” things in object oriented design. In fact we had overdosed on objects and had become full-blown oopaholics. Rico has a blog and video on the abuse of object-oriented design worth checking out (Rico was working on the compiler at the time I was working on the class library)

As a result we had a rude awakening. Our manager’s manager’s manager was getting frustrated with progress and we were under the gun. We had a big meeting to review our lack of progress and had to admit that we had not made 12 months of progress over the last year. We had to regroup.

I received my first Microsoft lesson: just because you messed up does not mean your career is over. What matters is how you dust off, what you learn, and if you can put that into practice. Microsoft holds no grudges. The boss said “ok you made a mistake, let’s get moving and don’t make that same mistake.”

We regrouped. We spent a couple of days talking about what we learned and very quickly came up with a number of “rules” about how we would really design a great class library for Windows. I can’t quite remember them all but some of them were:

  • We’re building a class library, not a compiler test suite, so there is no need to try to use all the features of C++.
  • Don’t build new abstractions on top of the existing abstractions in Windows. Folks will need to know Windows to use the framework anyway.
  • Developers will want to call Windows so don’t cache any state in the classes that is duplicating state maintained by Windows (for example, parent child window lists, styles, etc.)
  • Performance matters from the beginning so don’t go and use fancy stuff that has a fixed overhead that you don’t know if you need (like using virtual functions everywhere for example).

And so on. Once we had these lessons in place we actually developed the whole framework in a few months and were able to ship it with Microsoft C++ 7.0 (our first C++ product). We designed in parallel the next release along with all the graphical tools that became Visual C++, which itself became Visual Studio over time. The class library, Microsoft Foundation Classes or MFC, became a great tool used by tons of Windows developers around the world. I think my proudest moment was walking up to a really tall guy at a DEC booth in a partner section from the University of Ilinois at a tradeshow demonstrating an alpha of a Windows network application called a “browser”, I think his name was Mark, and asked him what tools he used and he said “MFC of course”. Yes!

When I think back about the dismal failure of our first library and how as a team we seemed to have developed a massive case of second-system syndrome for a first system (a mean accomplishment) and how we regrouped, learned lessons, and put those into play, I realized that without that first failure we never would have developed the success criteria that allowed us to build MFC.  

The fact that Microsoft offered an environment where we could practice the lessons detailed by Henry Petroski made the whole experience even better.

And most of all I am thankful that I got to be part of an amazing team who mentored and pulled me through these experiences as a new hire. I really want to thank all the people on this project that taught me so much. Thank you for letting me be part of our collective failure and success!

--Steven

PS: I used links to Amazon, but by all means buy your books from your favorite bookseller.