Abstractions: You can have too much of a good thing


Architects love abstracting things. And why wouldn’t they – it allows you to hide those pesky implementation details out of the way so they don’t trouble your callers, and lets you completely change the implementation at a later stage, provided the interface isn’t changed. But like most of the good things in life, there comes a point where more is no longer better.


All of the Enterprise Library application blocks include great examples of abstraction, generally through the use of the Factory and Plug-In patterns. An application using an Enterprise Library application block need only code against an interface or abstract base class, such as Database, AuthorizationProvider or Validator (or in some cases this is abstracted even further via a façade class such as Logger or ExceptionPolicy). The concrete implementation of each class is determined at runtime by a factory class, which will do things like inspect configuration files and attributes to figure out which one is appropriate. This pattern provides a lot of great benefits, as you can write code at an abstract level, such as “validate this!”, “log this!” or “call this stored procedure!”, while the specifics of how each of these is done can be encapsulated into a different class and changed without impacting your code.


Over the years I’ve spoken to a lot of people who wanted to (or actually did) take this a step further by completely abstracting away the use of application blocks by hiding them behind their own interfaces and factories. The main arguments I’ve heard for doing this are as follows:



  1. It provides an insurance policy just in case you ever need to get rid of Enterprise Library, by letting you build your own alternative implementation without requiring you to change your application code
  2. It lets you build reusable assets that can be used by different users who may or may not want to use Enterprise Library

I appreciate that these goals can be important for at least some people, however I’m far from convinced that abstracting away an Enterprise Library block is a good solution. David Hayden started a vibrant discussion on this topic around achieving this second goal in the Repository Factory, and continued it with a number of blog posts (here, here and here). Persistent though David is, I don’t buy his arguments. This is not because I think the blocks are perfect and that nobody would ever want to use anything different. However since the blocks already expose an abstracted interface, any further abstractions would need to expose an almost identical interface (or potentially a much watered-down subset). David didn’t share his example, but I’m assuming he built something like this for the DAAB:


public interface IDataAccessor
{
    int ExecuteNonQuery(string command, params object[] parameters);
    IDataReader ExecuteReader(string command, params object[] parameters);
    DataSet ExecuteDataSet(string command, params object[] parameters);
    // Add as many more DAAB-like members as you want
}

public static class DataAccessorFactory
{
    public IDataAccessor Create(string instanceName)
    {
        // Look up configuration and figure out what to return
    }
}

public class EnterpriseLibraryDataAccessor : IDataAccessor
{
    // Wraps an EntLib Database instance and defers interface calls to it
}


This solution meets David’s goals of breaking the dependency between the client code (such as the Repository Factory) and the Data Access Application Block. But let’s think a little more about what we’ve achieved here. We’ve replaced the DAAB dependency with a dependency on a new interface which is almost identical to the DAAB. Any arguments around why the DAAB dependency may not be desirable should apply equally to this new interface (plus we’ve added more complexity and more moving parts without adding any new functionality).


Even putting this argument aside, another problem I have with this approach is that I can’t really see anyone being able to build any other implementations of this interface that are different enough to be interesting. I’m not saying there aren’t other interesting ways of accessing a database – solutions such as NHibernate and LINQ to SQL are great examples. But these are philosophically so different to DAAB that they couldn’t possibly implement the same interface that was built primarily as a DAAB Abstractor.


The only way I can see to avoid this problem is to build an interface so abstract that it provides almost no functionality. Using Logging as as simpler example, one could build an interface like this:


public interface ILogger
{
    void Log(string message, string category)
}


…and build different implementations that could talk to the Logging Application Block or Log4Net. But we were only able to do this because we dumbed down the interface to the lowest common denominator. Both logging solutions use quite different approaches for things like routing, filtering, formatting, and these decisions influenced the design of their native interfaces. In our desire to provide a single interface that is abstract enough to work with both solutions, we’re going to lose a lot of functionality. I’ll wager that not many developers are willing to give this up for the theoretical goodness that this additional layer of abstraction provides.


To finish off this discussion, I wanted to add support to Chris Tavares’s observation that David’s problem is really a design-time one, not a runtime one, meaning that once you’ve decided what classes you want to use to talk to your database you’re unlikely to want to change your mind after deployment. While the abstraction patterns we’ve been talking about do allow for both design-time and runtime flexibility, Software Factories provide some additional options for design-time variability, such as code generation, that are probably more appropriate in this case. The nice thing about going this way is that at runtime you only have the code you need – and you have just the right amount of abstraction to provide a great flavour, and not so much to cause a stomach ache.

Comments (19)
  1. centur says:

    I think the actual problem arised here is about when people adding one more abstraction layer or hiding app-blocks behind the interfaces, they usually makes not well-thought interfaces.

    Most of time when coder want to hide something behind interface (with "books says to do that" motd banner), he just copy current methods signatures into the interface and continue working with it. It’s not adding anything to breaking dependencies because there is nothing about abstraction for current class to more abstract one. Its just another copy-pasted layer. Imo this is one of the differences between developer and coder – developer can imagine consequences of the code he writting.

  2. ploeh says:

    Tom, most of your arguments are quite valid, but there’s at least one good reason for abstraction away Enterprise Library:

    Until now, p&p have been releasing software faster than most enterprise organizations can keep up, and they don’t guarantee backwards compatibility (or at least: That’s the perception).

    Many development organizations have significant investments in software that uses a particular version of EntLib. As soon as this software has been released, it enters maintainance mode, which means that it will typically require great effort to change the version of EntLib (or so the organization thinks).

    Abstracting away such implementation details as EntLib (or Log4Net, or anything similar) will make it easier to move code in maintainance mode forward and keep it current.

    Yes, there’s a price to pay, but there are also benefits. In a single software project, it may not make sense, but it might when you look at the entire application lifecycle ecology of a larger organization.

  3. Vyas says:

    Tom,

    I agree with Ploeh here. In our organization we deal with clients who have their own application blocks/foundation services, few clients who use EL (multiple versions) and few clients who use other blocks like Log4Net. It makes sense for us as a huge organization to standardize on the interfaces for basic foundation services (Data access, Logging, Exception Handling, Authorization etc). Then create adapters for each of the current blocks. Eg Adapter for Log4Net as well as EL logging etc…

    Currently we have adapters for EL 2.0, EL 3.1 and few of our client frameworks. We have our own foundation services as well. This abstraction helps our huge developer community in our organization a single common interface. Our group works on providing the adapters which ensures to use the latest features of the changing application blocks ( EL 2.0 to EL 3.1 ). We do not break the interface. This would ensure that we have a pure plug and play option here. So, I think it does make sense in our situation. I do agree that we end up not supporting some fucntionalities at times but its worth the flexibility if offers for a huge organization like ours.

    Thanks,

    Vyas

  4. David Hayden says:

    Good post, Tom.

    If I always wanted to use Enterprise Library and being dependent on it wasn’t a big deal, I agree with you 100%. There is no reason to add another layer of abstraction in that case.

    Unfortunately, this is not my world. I don’t or cannot always use Enterprise Library and therefore I need more of a pluggable model around the data access in the Repository Factory. This pluggable model also insulates me from version changes in Enterprise Library, which has been a real issue for me when dealing with the software factories.

    This is why I chose to define a data access contract in my version and use the Adapter Pattern to isolate Enterprise Library as just a provider, not a dependency. My situation is very much similar to what Ploeh and Vyas are describing above.

    In my case, I am not adding an unnecessary layer of abstraction, but doing it very much based on the needs of my clients and their applications.

    I hope this at least helps clarify my position. As to whether this is important to do as part of the Repository Factory, perhaps not. I was only making a suggestion based on my needs and am not necessarily saying that these needs ring true for all the customers of the Repository Factory.

    Thanks,

    Dave

  5. Olaf says:

    Good post Tom, strong argument Ploeh.

    My 2 euro cents:

    Adding an abstraction can be justified in 2 scenario’s.

    1. I solve a problem, which I would like to make easier to consume.

    E.g.: "The people in my company find this library too complex to figure out, I’ll make this easier by adding another abstraction level". To me this is valid, even with EL.

    Remember, Enterprise Library is build for 80% of the scenarios, with the remaining 20%, there might be scenario’s or team’s in which EL is perceived as complicated to consume.

    2. I use a library someone else made, which I don’t want to take a binary dependency on and hopefully localize the change in case I find a library that does this job better (or upgrade to a new version for the matter of argument).

    David, this is where you are, right?

    I think this makes sense; thought a clean abstraction is very hard to come by. Following this argumentation, you would typically have to start with the consumer of the library to define the contract.

    In many cases (not all, say 80% :P), this type of abstraction is not worth the effort.. Since it not only implies creating a new contract for each consumer (or standardizing within an org.). This also implies you’d better think it through rather well, or else, after switching libraries you’d have something that smells of A, but actually behaves like B.

    Creating yet another "1-solution-fits-most-scenario’s" – abstraction seems like a big waste of time, to me. (assuming you are not re-targeting this to another audience).

  6. Olaf says:

    To that I’ll add:

    I would not consider "Software Factories" re-targeting to another audience. with "re-targeting" i was thinking more along the lines of DLR, CopmactFx, maybe an IoC application host, whatever.

    I also do not think it is possible to standardize the logging needs of Software Factories in general either.

  7. tomholl says:

    Great discussion everyone. Mark (pleoh) hit on an important scenario that I hadn’t really discussed – I see this as a variation on the "insurance policy" scenario, but since the goal is to abstract multiple versions of the same block, most of my earlier arguements don’t apply. While I can definitely see why people would go down this path, it still doesn’t sit that well with me. First, while p&p goes out of the way not to promise backward compatibility, in reality there were only a handful of breaking changes between 1.x and 2.x, and none between 2.x and 3.x, so "upgrading" an application to a new version of EntLib really isn’t a big deal even without an additional abstraction layer. Also I’d recommend applying the "don’t fix it if it ain’t broke" principle wherever possible, rather than insist that every deployed app uses the same version of EntLib (or any other shared components for that matter).

    But if we accept the fact that easier migration between versions is necessary, I think the ideal solution may be to version the block’s public APIs separately to the implementations, rather than build another interface around the blocks. If there were no breaking changes from one release to another, the interface assembly would not be touched. But while I think this approach could be quite nice, it would be quite a change from what we have today, and I don’t think many people would appreciate the irony of introducing breaking API changes for the sole purpose of preventing breaking API changes :-)

  8. Dru says:

    I started using EntLib at 1.x and have migrated many projects from one version to the next.  The only time that I had problems was from a version of 1.x customized in-house to work better with .NET 2.0 to 2.0 of EntLib.

    Even then, the dozens of projects migrated in a few weeks, not months.  Every version since has been an easy re-compile, re-test, release.

    I don’t buy that there is a need to abstract away EntLib in an organization to ease migration or enable use of multiple versions.

    I can think of only one case where it’d make good sense to put an abstraction in front of EntLib:  to replace the implementation of an interface that’s used pervasively in a existing code-base and only in the case that the code-base is sufficiently large or brittle.

  9. David Hayden says:

    Olaf,

    To answer your question, unfortunately, my situation is that I have some clients where I would like to use the Repository Factory ( and other software factories ) that do not or will not touch Enterprise Library.

    So, although Tom is focusing on the versioning piece, this was never my main point – it was just a nice bonus.

    My problem is that the Repository Factory as well as all the other software factories require Enterprise Library. This means that if I don’t or cannot use Enterprise Library, I am pretty much screwed due to that dependency.

    It is very much an unnecessary dependency in my opinion, but I can work around it. I have essentially created my own version of all the software factories that allow me to use something other than Enterprise Library if I need to.

    Obviously if I lived in an ideal world and everyone used Enterprise Library, I wouldn’t be wasting my time with the custom changes nor the request.

    The cool thing is that it is amazing how well you actually get to know the software factories when you need to re-write them :)

  10. tomholl says:

    But David, you haven’t eliminated the dependency on Enterprise Library – you’ve just moved it down a level. In order to make the factory work with something that’s not EntLib, you still need to build and test new code. Sure, you won’t need to recompile the data access layer to do this, but so what?

    I appreciate that there are users that can’t or don’t want to use EntLib, but I don’t understand why you’re dismissing the design-time solution (replacing or augmenting the code gen templates) that Chris suggested.

    Tom

  11. Steve says:

    I think David is spot on.  Having a Repository pattern in your application without dependencies on Ent Lib is not a bad thing.

    Seems to be an attachment to EntLib from it’s makers whereas the user community would see value in this.

    I know I would.  That is all that is needed.  +1 to David.

    To me, the same argument can be said about IoC – I would rather use Windsor in my applications.  I should be able to plug that into the EntLib or detach it without concern.

    Loose coupling is a positive thing  :)

  12. Jay R. Wren says:

    I think you are right to be concerned, but I don’t think that the answer is "dont’ do that"

    Take a look at Castle Project’s Castle.Core.Logging namespace and notice their version of ILogger is an abstraction for their own logger. Then take a look at their log4net and Nlog Services.

    It is really great to abstract away both log4net & NLog and use only Castle Core’s ILogger and simple log implementations. Then you can pull in log4net and/or NLog if you need them.

    IMO, it is a great example of abstraction done right.

  13. I agree with you Tom it’s a common law of leaky abstractions (Joel on Software).

    A perfect example is last week when I was helping a developer use the data access block in ent lib 2.0.

    A developer was making SQL stored procedure calls against an SQL Server database. When inserting data using parameters [ala db.AddInParameter(…)] all of a sudden his values were being inserted into the wrong column.

    After some searching around and prodding for more information the only thing he did was change the order of parameters, looking at his code for quick bit one could not see why this would fail (it was an sql server database he was connecting to after all, parameter order shouldn’t matter)…. enter the law of leaky abstractions…

    Since the database object isolated the developer from the database vendor and the code used to interact with the database was at such a high-level, the developer lost sight of the plumbing (the ADO.NET driver he was using) which caused a leak. The driver he was using was an older OleDB driver, which requires parameters to be in order. The program didn’t fail, there was no exception able to be thrown, a leak formed. A change of the provider to native .NET SqlServer driver fixed his problem.

  14. anothr user says:

    One new subscriber from Anothr Alerts

  15. Francois Tanguay says:

    The only things I tend to hide behind a custom interface are static facades.

    It helps with mocking/unit testing.

  16. Mark says:

    Tom,

    Great article. Another coworker and myself constantly run into disagreements with the "architects" where we work who insist on wrapping Enterprise library. Unfortunately these architects don’t code much, and don’t fully understand what EntLib is, so the new interfaces provided ultimately eliminate the flexibility of EntLib, for example the wrapper for EntLib logging requires my app code to specify my event sink(EntLib 1.x). ?????? WHY ???????  

    It was mentioned that sometime one would wrap EntLib to allow adapter pattern to be used in order to switch between Log4NET and EntLib.

    I view EntLib as an abstraction to common application services. I’m not at expert on all of EntLib but if I want to use a different logger, why not just build an adapter and plug it directly into EntLib via config.  Why can’t entlib be your interface to common application services. Can’t an adapter just be built to plug in to EntLib for the specific logging component you want to use? For instance Log4Net, or even to use a newer version of EntLib that has breaking changes(you obviously can make use of new features in EntLib, but possibly ease upgrade path).  Am I way off base here or does this make some sense?

  17. Joe says:

    What David Hayden has done with Repositories is fantastic.

    I’ve listened to his podcast, seen his examples.  I might actually start using ETLib now.

    I agree with Steve above, allow substituting Windsor in place of ObjectBuilder along with David’s work on Repositories and we’ll actually have software I’m interested in using.

  18. Joe says:

    By the way, this is a pattern that you can read about from Martin Fowler.  EntLib allowing Domain driven designs patterns is fantastic.

    (I’d like to see his Repository factory sit on top of NHibernate)

  19. Brian Johnston says:

    I couldn’t agree with you more Tom, in fact I think some of p&p stuff is too abstract itself.  I’ve been buried in bugs caused by abstraction upon abstraction whose sole reasoning was ‘what if’ scenerio’s.  And you get down to it and there’s no quantitative purpose behind all the extra layers that were introduced; it just cost the company a few extra hundred thousand dollars because a few developers thought it was cool.  But don’t get me wrong; abstraction is needed, coupling is bad, but like spices in a dish a little is good, a lot ruin the meal.

    So often I run across software that is so fricken bloated because of all these added interfaces because someday someone may up and decided to change the core architecture of the system without consulting the business or stakeholders who sponsor the project.  Bah.  If you’re worried about that level of change in you’re architecture then someone obviously failed to analyze, document, or enforce the strategic enterprise architecture requirements.  I’ve found most of these ‘problems’ where stuff just happens to change with the wind is because of lack of or poor project management or the project is run by coders who are hip to all the new technologies and want to keep up with change rather than properly retiring the current version and putting a new version in place…again strategic enterprise level requirements.  But yeah…requirements…who needs those right???

    I’m going to paraphrase something Martin Fowler said in his book ‘Refactoring’ – speculative generaliteis are when a coder adds something or some architecture because ‘we may need it someday’ – if it’s not needed it’s just adding extra complexity and maintnance costs.  On the side, it’s interesting that so many people who add so many layers of abstraction are big fans of Martin Fowlers Enterprise Patterns (or similiar books) and this is why they do it (because the book said so, which was said earlier), yet so few of them has read a book that is so much more important.

Comments are closed.