Changing Abstractions in an Object Model is Hard

Up until recently, I was working on a rather large spec for an as-yet-un-announced product. The majority of the spec consisted of class definitions and the details surrounding their interactions, but there was also a rather large conceptual component to the spec as well; you can't really dive into 50+ pages of object model specifications without knowing about the kinds of abstractions involved, how the different pieces fit together, and so on. I also spent time on a 30+ page threat model for the feature, but that's another story :-)

Anyway, I can't talk about this feature specifically (because it's not announced yet), but let's pretend the spec was for an object model for a car. When deciding how to design an object model for something complicated like a car, there are a lot of things you need to consider, such as:

·How accurately do you want to represent the modelled object?

·How easy do you want it to be for "typical" developers to use?

·How extensible does it need to be for "advanced" developers to customise?

·How performant / version-resilient does it need to be?

·And so on

In particular, you can probably tell right off the bat that the first two considerations are at odds with each other -- to accurately model a car, you would need to have objects representing every component of the car, perhaps even including the individual nuts and bolts holding the thing together (or, depending on the task at hand, maybe even the types of materials from which those nuts and bolts were fabricated). But to make a car object model easy to use, you want to simplify (abstract away) many of the details so that the developer has less concepts to understand (after all, they're a computer programmer, not an auto mechanic) and so that they can "discover" and use the features of the object model in a natural way.

For example, we'd rather write:

myCar.Color = Color.Red

than, say:

myCar.Body.Panels.Exterior.PaintJob.MajorColor = Color.Red

But on the other hand, we'd also rather write:

myCar.Tyres.Width = 18

than, say:

myCar.TyreWidth = 18

when you consider all the other properties of tyres that might be desirable.

So before you start designing the object model, you want to decide on your major abstractions (well, actually it's more of an iterative process, but anyways...). What details are necessary (the colour of the car), and what details can be glossed over (only the outer panels are painted)? What pieces of functionality can be logically grouped together (make and model of car), and what pieces need to be separated (car colour and tyre width)?

Now one of the abstractions we decided to make in this particular spec was something akin to saying that a car and its engine were one and the same thing. You couldn't have a car without an engine, and you could only have one engine in your car. Additionally, there weren't really any interesting properties of the engine (in our scenarios) so it didn't make sense to split off a separate Engine type. Nevertheless, there were still a few interesting things about engines that we needed to expose (like the Start method or the Stalled event) that we attached to the Car object.

Put another way, when speaking in plain English about their cars, people tend to say things like "I started my car" or "My car stalled the other day," rather than "I started the engine in my car" or "My car's engine stalled." Mapping object models to the way people think about The Real World is often more important than actually modelling The Real World.

So we have a design around one engine per car, and the world is a happy place. But then Someone Important says we have to support the new "hybrid" cars in our design -- those that have both an electric motor and an internal combustion engine (like the Toyota Prius or the Honda Civic Hybrid). Now, we've always known that hybrids existed, but we deliberately kept them out of our original design because (i) they are an edge-case scenario and (ii) they overly complicate the design for the rest of the users. (We like to have a "pay for play" approach at Microsoft -- you only pay the tax for advanced / complicated features if you want to use them. Never make the simple case overly complicated just to support some bizarre non-standard scenarios unless there's no other way around it).

So, we grudgingly add Hybrids to our model. And to be safe, rather than move from one engine to two, we abstract a bit more and say that you can have a collection of arbitrary types of engines with an arbitrary length. Need thirteen engines in your car? No problem. And we also need to make our concrete Engine class (which was an internal implementation detail of the old code) into a public IEngine interface that anyone can write a custom implementation of. Need an engine that channels psychic power into kinetic energy? No problem!

Now we can have a PetrolEngine class and an ElectricEngine class and stick them both inside a collection of IEngine objects inside a single Car object. Cool!

But there is a problem: what happened to the object model?

In the simple (non-hybrid) case, we go from:

myCar.Start()

to:

myCar.Engines[0].Start()

and in the more generic case we go to:

for each (engine in myCar.Engines)

engine.Start()

But realistically, what will happen is that developers will always use the first code snippet. Why? Because all the cars in their system will only ever have one engine, and the for each loop just looks ugly. Why loop through a collection that only has one element? And the program will work fine -- maybe for years -- until the first time someone plugs a hybrid into the system and it crashes [hah, a pun!] because the second engine isn't started.

The other (more important) thing is that we don't actually know how hybrids (or future variations thereof) will work. Do we really want to start both engines at the same time? Or do we want to just ask the hybrid to "start" and have the engines communicate with each other and decide when each one should kick in? What we need is another abstraction over the basic "collection of engines" that hides this detail. But since we don't know how the engines work, we must define some abstract interface that allows us (the car object model builder) to provide the developer (object model user) to deal with engines in an abstract way, whilst allowing the engine providers to plug in whatever features they need to make their engines work.

So now we have another new interface, IEngineCollection, that exposes things such as the Start method and Stalled event and provides some way for interested clients to dig deeper into the individual IEngine elements of the collection if they need to. We then provide a default implementation of IEngineCollection that supports a single petrol engine and simply forwards all the interface members on to that engine. We might call it SinglePetrolEngineCollection or some such thing, and have a nice easy way for clients to get an instance of it (or even just assume it's the one they want if they call the default Car constructor and don't supply their own implementation).

Manufacturers of hybrids can supply their own custom IEngine implementations (one for the petrol engine, one for the electric engine, one for the Mr. Fusion engine, etc.) and then their own IEngineCollection implementations that do clever things inside the Start method to decide which engine(s) to start, and so on. Then when users want to create hybrids they simply use the Car constructor that accepts a custom IEngineCollection implementation, and they're good to go.

Now to get rid of the "complexity tax" we can put back our original Car.Start method, which is just a simple wrapper that defers to Car.EngineCollection.Start, and the developer who just wants to start the stupid car can do so without worrying about multiple engines or about writing bugging code based on bad assumptions.

So why didn't we do this in the first place? We knew the problem existed, and there was a way to solve it that still followed the "pay for play" model.

One answer is that basically you don't want to make your system more complicated than it needs to be. Complications lead to bugs, and unnecessary complications lead to unnecessary bugs. Having lots of "pluggable" components (interfaces, abstract classes) makes it hard to reason about the system (and to threat model it!) because you can't be sure what any given extension will do. Allowing multiple, non-communicating developers to change the heart of the system also leads to reliability problems and other concerns that can't be tested or fixed by the OM developer (the AcmeEngine and the ContosoEngine both assume they have exclusive access to the fuel tank and don't co-ordinate their accesses to it, for example).

Another answer is that although we have hidden the complexity at a source-code level, we still have to expose it at the documentation level. The developer only needs to write myCar.Start to start her car, but if she ever looks at the documentation for the Start method she will be rudely yanked into a world of engine collections and layered abstractions and other such nonsense that she really has no desire to know about. Or, if she is just getting started with the object and starts to read the conceptual overview, there will be far too many things obscuring the basic design that she won't be able to see the forest for all the trees. This is a real concern, since documentation is incredibly hard to write and minimising the number of concepts a developer must grasp to use an object model is crucial to making it usable.

So I'm not sure what the point of this post was... ;-) Maybe it helps explain some of those really complicated Microsoft APIs where everything is an abstract interface and there are fifty different moving parts, none of which you really need to understand to use the API, but you're confused by them anyway.

Or maybe it just re-enforces the notion that you really should lock down your scenarios before you start designing, and then hope to %DEITY% that the Important People don't change their minds half-way through ;-)