The Model T and the Prius: Simplicity vs Complexity, yet again

My favorite conundrum, the difficulty of being simple, pops up everywhere I look these days. 

I had an epiphany shortly after buying a Toyota Prius recently:  The thing is an absolute Rube Goldberg machine under the hood, but is if anything simpler to operate than an ordinary car. By contrast, the Ford Model T was so simple in design that it came with all the tools needed to maintain and repair it, and a generation or two of backyard mechanics learned how to do just that.  It got 25 miles per gallon, better than Ford's typical product today. The Prius, on the other hand, is so backyard mechanic-unfriendly that the 12 volt battery is sealed away under the trunk; there is apparently no way even to use it to perform that one bit of auto repair that almost anyone can do, jump-start another vehicle.  But that complexity returns a big benefit - a Prius has all the interior size and environmental comforts of a conventional midsize sedan, but twice the fuel efficiency of the simple, uncomfortable Model T (and 3-4x that of Ford's current best seller).  The complexity is mostly hidden away from the driver; the only  leaks in the abstractions that make it look like an ordinary car are the tricks one can use to optimize fuel economy.

I noticed that Len Bullard recently made essentially the same point I am trying to make:

So it is with the modern combustion engine: it works, it is efficient with respect to the power delivered for the fuel and the weight of the overall vehicle, thus adding more efficiency, but it is not something that the average backyard mechanic can rebuild. The tools required, the parts required and the knowledge of how they work in combination exceed his or her resources, understanding and nerve.

Complexity is just as much in the nature of the evolution of systems as simplicity is desirable. At some point, the demands of an environment or a market require complex solutions. While simplicity is a goal, it can also become a religion just as harmful as fundamentalism when pursued with a sword. Complex systems can do what simple systems cannot do. The goal that all systems be accessible can be met with open standards, but the goal that they be powerful, workable and light might not even as the principles over which they are built remain the same.

Now Donald Norman, the guru of high design, weighs in:

“We want simplicity” cry the people befuddled by all the features of their latest whatever. Do they really mean it? No.

But when it came time for the journalists to review the simple products they had gathered together, they complained that they lacked what they considered to be “critical” features. So, what do people mean when they ask for simplicity? One-button operation, of course, but with all of their favorite features.

Joel Spolsky elaborates:

In the case of the iPod, the way beauty is provided happens to be through a clean and simple design, but it doesn't have to be. The Hummer is aesthetically appealing precisely because it's ugly and complicated.

I think it is a misattribution to say, for example, that the iPod is successful because it lacks features. If you start to believe that, you'll believe, among other things, that you should take out features to increase your product’s success. With six years of experience running my own software company I can tell you that nothing we have ever done at Fog Creek has increased our revenue more than releasing a new version with more features

Gadgetopia summarizes -- "you need features, but they need to appear  simple to the end users". 

There's a name for the discipline of working within a network of complex constraints to produce something simply usable - engineering.  It's not easy to get the complicated system of batteries, motors, charging systems, and drivetrain in a modern hybrid car to work together efficiently, but some very clever engineers managed to do that.  Those who dismiss various XML technologies, or XML itself, because of ugly complexities or unpleasant constraints may someday look as prescient as the folks in the auto industry who killed off the electric car in the 1990's, only to be humbled by the engineers who made the hybrid system practical a few years later.

OpenXML, WS-*, XSD, and XML for that matter are not doomed because they are complex, nor are they destined for success because they attempt to hide their complexity from the end user.   Those who build infrastructures and applications on top of them are challenged to do good engineering to make their purported features actually work and to appear simple to the ultimate consumers.  In the long run we'll probably end up more or less where automobiles are now - complexity under the hood that would inspire Rube Goldberg, but engineering quality that makes it appear simple to the driver.

Reasonable people can disagree on whether we are currently on the right track to making this happen.  If we are, the yelping in the blogosphere might be dismissed as backyard mechanics lamenting the fact that "enterprisey" applications can't be built with hand tools anymore.  If the critics are right, we could plausibly be in the midst of a disruption akin to the 1970's consumer revolt against Detroit-made gas guzzling behemoths in favor of smaller, better-designed imports. We shall see, but the answer is not at all obvious despite various "Mission Accomplished" declarations by one faction or another.

 Here what  I (personally ... obligatory disclaimer about not speaking with the xmlteam hat on) think about the technologies listed at the top of this post with respect to the simplicity/complexity dilemma:

OpenXML document format vs the Open Document Format? Nobody wins, neither loses.  Specialized libraries and hard work will make OpenXML appear simple enough for most purposes; the underspecification in ODF (e.g. for spreadsheet formulas) will be remedied with spec revisions that add complexity. Actual users will seldom know or care about the underlying formats because modern office apps are already doing a decent job of making all that stuff look simple.

W3C XML Schema vs RELAX NG? There's lots of room for evolution here; neither will win in the long run, but something will emerge that steals ideas from both.  XSD 1.1 already addresses some of 1.0's  more egregious flaws and incorporates a lot of ideas from Schematron.  In the "2.0" timeframe I'd expect a hybrid schema language that incorporates the best ideas from all three (although I don't know which organization or company that will come from).  It will be as "complex" as XSD measured in features, spec size, etc. but more like RElAX NG in terms of formal underpinnings and readability.

WS-* vs REST?  We'll end up with a composite Web/Enterprise architecture that has simple REST-like concepts at the core that will continue to be used by "Web 1.0", but can be more cleanly (than the current WS-* stack) extended to allow multi-hop / multi-protocol reliable messaging, security protocol negotiations, etc.  Even if the WS-* stuff disappeared, someone would quickly reinvent something very similar because the REST primitives are just too primitive to appear simple to any but the most zealous backyard mechanics.

XML vs JSON? I'm still undecided about this.  I'm feeling just a bit smugly prescient these days because I was once a flamethrower in the "XML is too complex" wars, and people are finally voting with their feet against XML's complexity.  On the other hand, XML has gotten so pervasive now that I don't see how JSON can achieve success as a data interchange standard (outside the AJAX realm) by the classic disruptive route of  competing with non-consumption.  It would be lots of fun to reinvent a JSON-based stack-o-stuff to give it a schema language, transformation language, query language ... not to mention a "LINQ to JSON" API!  I'm not so sure that our customers would find it fun to endure that disruption, however.  A well known and vocal colleague [hint: he likes to remind me that DOM means 'dumb' in Dutch] thinks I'm a corporate drone wimp  :-) for that view ... what do you think?