Changing Abstractions in an Object Model is Hard

Up until recently, I was working on a rather large spec for an as-yet-un-announced product. The majority of the spec consisted of class definitions and the details surrounding their interactions, but there was also a rather large conceptual component to the spec as well; you can't really dive into 50+ pages of object model specifications without knowing about the kinds of abstractions involved, how the different pieces fit together, and so on. I also spent time on a 30+ page threat model for the feature, but that's another story ๐Ÿ™‚

Anyway, I can't talk about this feature specifically (because it's not announced yet), but let's pretend the spec was for an object model for a car. When deciding how to design an object model for something complicated like a car, there are a lot of things you need to consider, such as:

·How accurately do you want to represent the modelled object?

·How easy do you want it to be for "typical" developers to use?

·How extensible does it need to be for "advanced" developers to customise?

·How performant / version-resilient does it need to be?

·And so on

In particular, you can probably tell right off the bat that the first two considerations are at odds with each other -- to accurately model a car, you would need to have objects representing every component of the car, perhaps even including the individual nuts and bolts holding the thing together (or, depending on the task at hand, maybe even the types of materials from which those nuts and bolts were fabricated). But to make a car object model easy to use, you want to simplify (abstract away) many of the details so that the developer has less concepts to understand (after all, they're a computer programmer, not an auto mechanic) and so that they can "discover" and use the features of the object model in a natural way.

For example, we'd rather write:

myCar.Color = Color.Red

than, say:

myCar.Body.Panels.Exterior.PaintJob.MajorColor = Color.Red

But on the other hand, we'd also rather write:

myCar.Tyres.Width = 18

than, say:

myCar.TyreWidth = 18

when you consider all the other properties of tyres that might be desirable.

So before you start designing the object model, you want to decide on your major abstractions (well, actually it's more of an iterative process, but anyways...). What details are necessary (the colour of the car), and what details can be glossed over (only the outer panels are painted)? What pieces of functionality can be logically grouped together (make and model of car), and what pieces need to be separated (car colour and tyre width)?

Now one of the abstractions we decided to make in this particular spec was something akin to saying that a car and its engine were one and the same thing. You couldn't have a car without an engine, and you could only have one engine in your car. Additionally, there weren't really any interesting properties of the engine (in our scenarios) so it didn't make sense to split off a separate Engine type. Nevertheless, there were still a few interesting things about engines that we needed to expose (like the Start method or the Stalled event) that we attached to the Car object.

Put another way, when speaking in plain English about their cars, people tend to say things like "I started my car" or "My car stalled the other day," rather than "I started the engine in my car" or "My car's engine stalled." Mapping object models to the way people think about The Real World is often more important than actually modelling The Real World.

So we have a design around one engine per car, and the world is a happy place. But then Someone Important says we have to support the new "hybrid" cars in our design -- those that have both an electric motor and an internal combustion engine (like the Toyota Prius or the Honda Civic Hybrid). Now, we've always known that hybrids existed, but we deliberately kept them out of our original design because (i) they are an edge-case scenario and (ii) they overly complicate the design for the rest of the users. (We like to have a "pay for play" approach at Microsoft -- you only pay the tax for advanced / complicated features if you want to use them. Never make the simple case overly complicated just to support some bizarre non-standard scenarios unless there's no other way around it).

So, we grudgingly add Hybrids to our model. And to be safe, rather than move from one engine to two, we abstract a bit more and say that you can have a collection of arbitrary types of engines with an arbitrary length. Need thirteen engines in your car? No problem. And we also need to make our concrete Engine class (which was an internal implementation detail of the old code) into a public IEngine interface that anyone can write a custom implementation of. Need an engine that channels psychic power into kinetic energy? No problem!

Now we can have a PetrolEngine class and an ElectricEngine class and stick them both inside a collection of IEngine objects inside a single Car object. Cool!

But there is a problem: what happened to the object model?

In the simple (non-hybrid) case, we go from:




and in the more generic case we go to:

for each (engine in myCar.Engines)


But realistically, what will happen is that developers will always use the first code snippet. Why? Because all the cars in their system will only ever have one engine, and the for each loop just looks ugly. Why loop through a collection that only has one element? And the program will work fine -- maybe for years -- until the first time someone plugs a hybrid into the system and it crashes [hah, a pun!] because the second engine isn't started.

The other (more important) thing is that we don't actually know how hybrids (or future variations thereof) will work. Do we really want to start both engines at the same time? Or do we want to just ask the hybrid to "start" and have the engines communicate with each other and decide when each one should kick in? What we need is another abstraction over the basic "collection of engines" that hides this detail. But since we don't know how the engines work, we must define some abstract interface that allows us (the car object model builder) to provide the developer (object model user) to deal with engines in an abstract way, whilst allowing the engine providers to plug in whatever features they need to make their engines work.

So now we have another new interface, IEngineCollection, that exposes things such as the Start method and Stalled event and provides some way for interested clients to dig deeper into the individual IEngine elements of the collection if they need to. We then provide a default implementation of IEngineCollection that supports a single petrol engine and simply forwards all the interface members on to that engine. We might call it SinglePetrolEngineCollection or some such thing, and have a nice easy way for clients to get an instance of it (or even just assume it's the one they want if they call the default Car constructor and don't supply their own implementation).

Manufacturers of hybrids can supply their own custom IEngine implementations (one for the petrol engine, one for the electric engine, one for the Mr. Fusion engine, etc.) and then their own IEngineCollection implementations that do clever things inside the Start method to decide which engine(s) to start, and so on. Then when users want to create hybrids they simply use the Car constructor that accepts a custom IEngineCollection implementation, and they're good to go.

Now to get rid of the "complexity tax" we can put back our original Car.Start method, which is just a simple wrapper that defers to Car.EngineCollection.Start, and the developer who just wants to start the stupid car can do so without worrying about multiple engines or about writing bugging code based on bad assumptions.

So why didn't we do this in the first place? We knew the problem existed, and there was a way to solve it that still followed the "pay for play" model.

One answer is that basically you don't want to make your system more complicated than it needs to be. Complications lead to bugs, and unnecessary complications lead to unnecessary bugs. Having lots of "pluggable" components (interfaces, abstract classes) makes it hard to reason about the system (and to threat model it!) because you can't be sure what any given extension will do. Allowing multiple, non-communicating developers to change the heart of the system also leads to reliability problems and other concerns that can't be tested or fixed by the OM developer (the AcmeEngine and the ContosoEngine both assume they have exclusive access to the fuel tank and don't co-ordinate their accesses to it, for example).

Another answer is that although we have hidden the complexity at a source-code level, we still have to expose it at the documentation level. The developer only needs to write myCar.Start to start her car, but if she ever looks at the documentation for the Start method she will be rudely yanked into a world of engine collections and layered abstractions and other such nonsense that she really has no desire to know about. Or, if she is just getting started with the object and starts to read the conceptual overview, there will be far too many things obscuring the basic design that she won't be able to see the forest for all the trees. This is a real concern, since documentation is incredibly hard to write and minimising the number of concepts a developer must grasp to use an object model is crucial to making it usable.

So I'm not sure what the point of this post was... ๐Ÿ˜‰ Maybe it helps explain some of those really complicated Microsoft APIs where everything is an abstract interface and there are fifty different moving parts, none of which you really need to understand to use the API, but you're confused by them anyway.

Or maybe it just re-enforces the notion that you really should lock down your scenarios before you start designing, and then hope to %DEITY% that the Important People don't change their minds half-way through ๐Ÿ˜‰

Comments (9)

  1. Keep up these cool articles!

  2. AndrewSeven says:

    What about using the Engine as an abstraction of motors.

    From the view of the driver, the car is powered.

  3. Peter Torr says:

    Andrew, that is essentially what this does (unless I misunderstand your comment ๐Ÿ™‚ ). IEngine is an abstraction of an engine, IEngineCollection is an abstraction of a set of motors (the "power" for the car) and the Car.Start method is a nice wrapper to hide the details from someone who doesn’t care.

    IEngineCollection could itself be an IEngine, by the way.

  4. AndrewSeven says:

    I was thinking of keeping the name Engine instead of EngineCollection.

    An Engine could have an IMotorCollection of IMotors.

  5. Peter Torr says:

    Indeed. It’s just a matter of spelling / OM depth at that point ๐Ÿ˜‰

  6. S N says:

    The problem in the real life is that people always substitute "How a service is provided" for "What service is needed".

    Since "How a service is provided" always has both information of what and how, it is easy to convey two facts in one sentence.

    Where as the abstraction mechanism separates them into two clearly marked objects with well defined boundaries and clearly defined interacton between them.

    It is always easy if they are treated two different independant objects from the very begining.

    The problem araises only when the abstraction is derived from how-what in to a how and what case.

    The derived abstraction is always prone to create its own legacy in the system for ever.

    Your case falls in to a second one. Since it is a derived one, it has prone to legacy issue and sure its has one hell a lot of issue in it.

    The issue in your case is explained in detail below.

    The application needs to know what a car engine has(can) to do?

    * Start

    * Stop

    * Accelarate

    * Decelarte

    * Stalled

    and so on.

    Here, what the consuming application doesn’t need to know is how these services are provided by the engine.


    Though the mechanic application needs to know how the engine is built so that it can provide repairing services.

    But both applications are come different under dimension of abstraction.


    What you have solved so far in this case is to just increase the scope of abstraction and yet it failed to provide the perfect but simple abstraction that is required.


    Your new abstraction goes further one level to take care of multiple engines but still, it is in the form of how-what rather than how and what separately.

    Because it assumes that all the engines would be working in the same level.

    What if an engine is made to control multiple smaller engines that togather presents an aggregated view of one single engine. The consuming application doesn’t care this how though the mechanic application needs to know.

    If i put it in a simple form, the engines can be structured in a tree like fashion.

    Your abstraction doesn’t take care of this.

    Why the comsuming application needs to know about how the engine is constructed.

    The ideal case for the consuming application is that you should have only one interface called IEngine. No IEngineCollection since it is in how-what format.

    How the IEngine is implemented is for the engine manufacurer’s worry and not the consumer’s problem.

    The other problem your talking about is upward inheritance. That is, the parent will inherit its child’s attributes at runtime as longs as the parent can distinguish from whom these attributes are inheritted.

    In a typical case at runtime, the child will push its attribute to its immediate parent. The immediate parent will inherit them as long as they are unique. After inheritting its descendant’s attribute, the parent will push the new colection of its attributes (its own plus descendant’s attributes) to its immediate parent. This cycyle completes when the root node of the tree is reached.

    This process can be applied only when the object-type graph is a perfect tree.

    Finally, kudos to this blog for helping people understand how the abstraction should be done.

  7. Peter Torr says:

    Thanks S N,

    I agree that IEngineCollection should also "be" an IEngine (as hinted above). This allows exactly the tree structure you describe.

    (Since this was just an anology for something I can’t talk directly about, in our particular instance there could never be sub-collections in the "tree," but in an unrelated part of the API we do use exactly this approach – IThingCollection is also an IThing so you can chain them as deeply as you want. Even then, it is unlikely you would evere do so due to the nature of the beast).

    The reason IEngineCollection *must* be specified though is because the people providing the engines and the people providing the "mechanic" services are different people. Without Microsoft specifying what interface they should use to talk to each other, they will have to deal with multiple vendor-specific concrete classes (AcmeEngineCollection, WidgetEngineSet, etc) which becomes a nightmare for people to support and leads to incompatibility.

    The general user of the Car object never needs to know this level of detail, but (eg) the CarMechanic class does.

  8. Sometimes less experienced engineers err on the side of flexibility. We have probably all seen a complex object model, rapidly created, that is extemely flexible, but solves none of the most important problems that need to be solved. (Whoops! I ran out of time during implementation).

    Flexibility in the form of interfaces is one way to try to compensate for incomplete knowledge or experience in the problem domain. But it never really works. Without an expert in the domain, that may not be a software engineer, telling designers exactly how people want to use the software, most likely lots of flexibility will be added, creating a complex API that misses the sweet combination of power and simplicity.

Skip to main content