The value of Intermediation in SOA

I find myself surprised.  I guess I had assumed that everyone has already read books like Enterprise Integration Patterns by Hohpe and Woolf and Patterns of Enterprise Application Architecture by Fowler.  I guess I was being optimistic.

SOA in an architecture for Enterprise Application Integration.  If integration isn't part of the goal, SOA is sound and fury signifying nothing. (my apologies to the bard).

In SOA, there is a very specific principle that intermediation gives us: substitutability.

A little Context: the Factory Pattern.  When an interesting article was published in DDJ a number of years ago titled "Use of Java New considered harmful" it raised a good bit of noise.  After all, how could it be 'bad' to use the 'new' keyword to create an object.  The author correctly pointed out that creating an object binds the code in the module to the type itself.  It is impossible, if you use the language keyword, to create a subtype based on configuration if you use 'new'.  This was part of the general move towards patterns, a move that brought us dependency injection and the Spring framework, to name a few.  The value of these innovations has been tremendous.  In systems where they are used, dependency injection patterns lead to tremendous gains in productivity in the code.

The principle that supported this: Liskov Substitution Principle. 

In order to make this change, factory patterns became commonplace.  While I do NOT recommend that every object be created with a factory, I do recommend that any object that is to be considered reusable or extendable be created with a factory.  Otherwise, reuse or extensibility is a fiction (a chimera, as one of my respondants mentioned on a prior post).  Over the years, we changed the practice of OO development.  "Decouple fiercely" is a good way to describe the change.  You decouple the code every time unless you can justify NOT doing it.  Big change.

At the architectural level, however, we didn't look at the substitutability principle in an OO manner.  Perhaps that is because many of the principles of OO, including inheritance, are quite a bit harder to grapple with when you are dealing with components.  Hiding a component is not the most positive thing to do when you have to deploy it anyway.  :-)

Instead, we took the same notion: "decouple fiercely," and we created the concept of services.  They are decoupled... fiercely decoupled.  We use one protocol to communicate.  We can intermediate anything.  We have one mechanism to monitor and manage.  This led to the growth of ESB patterns and entire product lines that move information using messaging.  It is a sea change.

But no good deed will go unpunished, and no good idea will be remembered if it is not justified every now and then... so here goes.

The ability to add intermediation gives me some fairly specific advantages.  Just like factories give me the benefits of the Liskov Substitution Principle in OO design, intermediation gives me the benefits of substitution in messaging design.

Intermediation is the ability to observe a message passing from one trading partner to another and to take action based on the contents of that message (assuming I have been given proper access to it). 

Those actions include

1. changing the destination to another system,
2. changing the destination to two systems (useful when transitioning from one destination to another),
3. allowing events to be published from multiple sources,
4. adding event handling and orchestration as the result of the business event,
5. repurposing or repackaging the event for exposure on a different protocol or externally to a public interface like SaaS would require. 

These are the tactical changes.  Consider the actual abilities this introduces.  For example, #4: adding orchestration.

Let's say that system 1 generates an invoice.  It sends an event to the world saying "invoice here" and system 2 captures that message.  System 2 asks for details about the invoice... perhaps it will place the information on a web site for internal support teams.

Let's say that we are moving to a CRM solution in our internal support groups.  We want to create the information in the CRM system related to the invoices that specific customers have been issued.  We need to integrate these two systems.  The existing web app needs to have a link to the CRM system's data, to allow the user to move across easily.

We can intercept the request for further information from the web app to the publisher.  When the publisher responds with information about the invoice, we can insert the invoice in the CRM system, add a link to the CRM record for that invoice to the data structure, and resume our response to the web app.  Assuming that our canonical schema has a field for 'foreign key', we have just integrated our CRM and web information portal... without changing either one.

Clearly, we'd still want to add a front end bit of functionality to allow the link to be transversed, but the data integration is done... without either the publisher or the subscriber having to change.

This is just one example of intermediation in practice.  I'm sure that MANY folks can present other examples.