The Unimportant SOA Catalog

Have you ever woke up in the morning with an idea in your head that you simply have to write down?  I just did.  Here's the idea: Everyone talks about how important the catalog (or repository) is to Service Oriented Architecture.  It isn't.

The reason everyone wants a catalog is simple: If I create a uniquely valuable service, and I want people to use it, I need a place to advertise it.  So I put it in a catalog.  The catalog contains useful information like what the service is, and what it does, and who made it, and how to call it.  Useful stuff.  Sears Roebuck, circa 1893.

So how can that be unimportant?

Because this is a case of 'doing a really good job of solving the wrong problem.'

A friend of mine and fellow architect here in Microsoft IT named Mohamed El-Ghazali changed the way I think about service contracts.  And Gartner changed the way I thought about "what makes adoption work" and together, there's a powerful brew.  It took me a while, because these ideas are "just different enough" to make me pause, but between these two sources, they had the intended effect, and now I can say, without blinking, that the catalog is not the high order bit.

Why? Because the catalog is not an IFaP.  It is a list of chaos. 

If you have 20 services, or even 50 services, a catalog is really useful.  I'm looking at an architecture that will require something around 500 enterprise information, activity, and process services, about 200 infrastructure services, and countless 'point solution' services.  There is no way a list will do.  No human can remember it, or use it.  Duplication and overlap will prevail.  Face it, the catalog doesn't scale.

So where does the solution lie?  

How about looking to the past to find the future. 

I call your attention to the history of the Periodic Table of Elements.

If you are not familiar with the history of the creation of this simple yet extraordinarily powerful concept, you should read this page.  Two key concepts I'd like to pull out:

First off, by creating the periodic table of elements, Mendeleev created a situation not only where elements could be classified, but where missing elements could be predicted.

Between 1868 and 1870, in the process of writing his book, The Principles of Chemistry, Mendeleev created a table or chart that listed the known elements according to increasing order of atomic weights. When he organized the table into horizontal rows, a pattern became apparent--but only if he left blanks in the table. If he did so, elements with similar chemical properties appeared at regular intervals--periodically--in vertical columns on the table.

Mendeleev was bold enough to suggest that new elements not yet discovered would be found to fill the blank places. He even went so far as to predict the properties of the missing elements. Although many scientists greeted Mendeleev's first table with skepticism, its predictive value soon became clear.

This meant that not only did Mendeleev help to understand the list of 'needed domain knowledge', he actually created boundaries that empowered other people to focus their efforts and deliver incredibly quick innovation.  This innovation came from people he had never met.

The second thing I'd like to highlight is that the original table was useful but it was changed as knowledge increased to match a more modern understanding of chemistry and modern techniques for measuring atoms that was not available when it was developed.  In other words, the concept is good, even if the implementation is iterative.  (19th century agility).  The boundaries remained, and the table stands today as a fundamental artifact in the understanding of our natural world.

What does that have to do with SOA?

I am creating a similar table of services based (loosely) on the layers defined by Shy Cohen, message exchange patterns defined by the W3C, the work on Solution Domains that my team in IT Enterprise Architecture has started, and the business behaviors that I see as necessary to accomplish a partitioned design.  The goal is to create an all-up IFaP of services based on multiple spanning layers. 

Unlike the periodic table, this will not be bounded by physics.  Instead, it will be bounded by the data elements and solution elements defined by performing a Solution Domain mapping exercise against the enterprise.  Your organization will have different elements, but either way, there will be boundaries, and that will, I believe, foster organized and directed effort, creativity, and discoverability.

I believe the value will be clear.

1. We will know what services we need to develop to meet the needs of the enterprise.  We can even prioritize the list and create a roadmap showing the CIO when we will be "done."
    
2. We will have basic patterns already established for how they will be called and what they will return.  This reduces a huge amount of churn and will give brave developers the ability to resist the "not invented here" plague.  The patterns can be designed to include all the needs of the test and support teams that are normally 'left out' of application specs but are ever more critical to the success of SOA.
    
3. We will have generic test harnesses in place to test them before they are written, allowing test architects to build reusable test value, while at the same time relieving project teams from writing difficult and complex test software to support SOA.
    
4. We will have sufficient information to estimate their cost by the team that must build and maintain them, providing some visibility to the cost of developing an integrated application.  This gives us the ability to seperate out the incremental cost of SOA from the cost of application development in general.

I'm pretty excited about doing this, and I think it is a strategy that can work. 

So what part of this kills the catalog?

The catalog helps a programmer to find the name of a service that performs a specific purpose. 

However, if I know the purpose, and the list of activities is a constrained list (as is the list of data subject areas), then I can create the name of the service and just hit the infrastructure up for it.  If it exists, the service will respond with details.  If not, the infrastructure can respond with information on what is needed and where it should live.

It really is that simple. 

We go from this:

The catalog describes the service infrastructure (bad)

to this

The catalog is the service infrastructure. (good)

And in this world, the catalog is informative, but not required.