Microsoft’s Enterprise Service Bus (ESB) Strategy – Part 2/4


About: This post is the second in a series of four about Microsoft’s strategy for Enterprise Service Bus.  This post discusses the various definitions of the term ESB, defines the attributes of an ESB and tries to dispel some myths about it


 


Greetings,


 


If you want seamless integration between systems implemented on disparate technologies, if you want to plug legacy systems in an SOA environment, if you want to have composite applications based on multiple services, if you want dynamic discovery and binding capabilities, if you want true interoperability, in short if you want to resolve all of your integration nightmares, then ladies and gentlemen I give you the ENTERPRISE SILVER BULLET (ESB).


 


The ESB is a magical and nebulous piece of software that plugs itself in the middle of your enterprise with neat bidirectional arrows going to boxes labeled ‘mainframe systems’, ‘Java Systems’, ‘.NET systems’ and ‘other legacy assets’, it is all you will ever need to reduce costs, increase your time to market and optimize your IT operations etc. etc. and etc.


 


As all of you know, despite sounding familiar, most of what I have written is incorrect or minimizes the amount of effort involved in achieving the desired outcome. Unfortunately most of what I have written is based on the descriptions and presentation of ESBs I have come across.


 


Let us explore a the brief history of ESB and beyond-the-hype attributes of this type of software


 


A brief history of ESB and its many definitions


 


The term ESB was first used by Sonic in 2002 to refer to their SonixXQ product that was an XML-enabled MOM product, the product was later re-branded as Sonic ESB. A few months later, Gartner called ESB a strategic investment and then very soon every thing from integration servers to messaging products were re-branded as ESBs, I even attended a presentation where a vendor showcasing their SOA offerings displayed the Ethernet wire as their official ESB strategy.


 


In addition to the re-branding there was also a plethora of definitions put out, everything from Gartner’s “A Web-services-capable infrastructure that supports intelligently directed communication and mediated relationships among loosely coupled and decoupled biz components”, to Burton’s group “The ESB label simply implies that a product is some type of integration middleware product that supports both MOM and Web services protocols” to IBM’s “To put it bluntly: If you have WebSphere MQ and other WebSphere brokers and integration servers, you have an ESB (Bob Sutor, IBM).”


 


In general, the market quickly realized the potential of promising a new pill for what was essentially an integration problem. In many cases the companies who bought the ESB products were disillusioned by the amount of effort and customization it takes to ‘plug’ a new system in to an ESB, the licensing costs involved in having an ESB node license in a distributed model, the learning effort involved in creating customizations/extensions for ‘plugging’ legacy systems and the maintenance nightmares. Ironically, in many cases businesses ended up deploying multiple ‘ESBs’ and once you have multiple ESBs what do you need to integrate them?……. Yes you guessed it right; you need another ESB to integrate the ESBs, one BUS to rule them all!


 


I am not against ESB as an idea, product or integration pattern, I am against setting unrealistic expectations around a buzzword that has resulted in to considerable grief for many early adopters.


 


Let us explore the basic attributes of this type of software


 


Attributes of an ESB


 


There are some attributes of an ESB that all major industry experts have agreed upon, an ESB at the very minimum should support:


 



  1. Brokered communication:  The core purpose of an ESB is to act as an intermediary between systems that need to send and receive data from each other. The notion of an intermediary is essential to the ability of an ESB to provide address indirection, rules-based routing, message transformation and other capabilities. The core value of having brokered communication is to increase loose coupling between systems and externalize integration logic for potential reuse and ease of maintenance

 



  1. Routing: In order to integrate systems in a loosely coupled manner, an ESB typically has some type of name space or directory which resolves a logical service name to a specific implementation at run time. An ESB should also be able to maintain and execute rules for topic-based, property-based or content-based message routing.

 



  1. Basic Web services:  An ESB should at the minimum support basic Web Services standards and their foundation, i.e. it should provide support for WSDL, SOAP over HTTP, TCP/IP and XML

 



  1. Endpoint metadata: An ESB must be capable of storing or accessing information about service interfaces and message schemas. 

 



  1. Non-functional requirements: An ESB should be able to support deployment in a high availability configuration; it should also be scalable and reliable and offer management capabilities

 


At the very minimum, any decent ESB product will include the above five properties as they form the core value proposition for buying a piece of software that eases integration and interoperability problems in an organization.


 


In addition, there are other attributes that lend themselves to be placed in an ESB, whether an organization would need some or all of these attributes will differ based on a variety of factors that I plan to discuss in my next post. Some of the other attributes that could be considered with respect to an ESB include


 



  1. Externalizing business processes and rules.   An ESB could contain capabilities that allow for externalizing business process and rules from applications and placing them in the ESB.

 



  1. Business Activity Monitoring. An ESB is a natural place to conduct business activity monitoring, if it is indeed the middleware through which systems are communicating, interacting and interoperating with each other then having capabilities that allow for analysis of the data for various stakeholders can be quiet useful

 



  1. Interoperability and integration.  Web Services are increasingly the ideal choice to implement interoperability between different systems, however, Web Services are sometimes not the best or the most optimal option. An ESB should ideally contain capabilities to facilitate interoperability without Web Services reducing the amount of custom effort required for each type of system.

 


Based on the above attributes, you can think of an ESB as


 


Enterprise: The “Enterprise” in ESB refers to the fact that an ESB will generally be installed and managed within one virtual enterprise – one company and possibly some of its customers and suppliers. 


 


Services: The “Service” in ESB refers to the value added services provided by the ESB, including routing, management and transformation.


 


Bus: The “Bus” in ESB refers to its support for non-disruptive service substitution. A logical bus topology facilitates these properties as a bus enables any endpoint to talk to any other endpoint in a decoupled manner.


 


Conclusion


 


In this article I discussed the history and basic definition of the term ESB. In my opinion rather than buying an ESB and retrofitting your needs around it, you should define what your organization requires from an ESB right now and what you will need from it in future. Based on the needs you can come up with an evaluation criteria that will allow you to choose an ESB (as a product or a pattern, something I will discuss in the next post) that provides the maximum value to your organization.


In the next post I will attempt to describe Microsoft’s strategy for ESB and discuss the future of this type of software based on the new developments in the industry around interoperability and integrations standards.


 


Best regards,


Mohammad

Comments (5)

  1. In short: BizTalk 😉

  2. makif says:

    yes you guessed it right, I am setting it up to make a case for Biztalk :-)…. Just kidding…actually I do not believe that Biztalk is always the right answer for implementing an ESB, you will need to stay tuned for part 3 and 4 but in short, no I do not intend to mock all the rebranding and then rebrand Biztalk as the ESB

  3. connector says:

    Great blog series on this, cant wait for the next parts.

    From my view point ESB’s are akin to AJAX. Its just a term given to a collection of tools to meet your objective. If you look under the hood or behind the scenes these tools have been around for a while. Its the term thats new.

    Companies like Sonic coined the term I think to try and productize a collection of patterns to try and solve the enterprise integration issues of selling into that market space. Its easier to sell something if you can get a customer to see or have them feel like they have a shrink wrapped product. ESB branding gives that appearence.

    Unfortunately like a bad christmas many may feel that the batteries were left out of the box.

    So I agree with your conclussion. Evaluation is really what you need. Afterall you may find you only need the ethernet cable and not the whole box of tricks and its associated bagage….

    Unfortunately with many companies hanging on the ESB brandingm Its hard to pull these components apart, causing the excessive licensing costs and also sometimes forcing the customer into an implementation pattern that may not be right for them or their environment.

    My advice pick and choose where possible and dont use the typcial ESB boundaires to try and force a decission.

  4. Hans Valcke says:

    Hi, Mohammed , great post

    I’ve tried to figure out what all the fuzz is about and I’m very confused at this time.

    We currently have a fairly large Biztalk env. in our company – which by the way works very well – and I’m trying to understand the ESB concept.

    I know that Biztalk is a hub and spoke but over the years we created services (which we call the client connector) that pickup messages (flat file, XML, CSV, …) and send them to the hub. The services are configured dynamically with webservices and we use ac/nack while sending them over HTTP or HTTPS. We can write adapters for these services, we have SQL stored proc adaptor so save data in SQL, we can read and write MSMQ, FTP … we can write as may connectors as we want. We also have error handling etc. by sending errors back to the hub(s)

    Moreover we also build a tool to monitor all the hubs (Biztalk) through webinterface and business users can correct problems in the integration layer when they have rights to do so. And because we use biztalk as a hub all interfacing in central (we wanted to move away from point to point interfacing).

    We have supppliers and customers which install the client connector to send orders, invoices etc.

    Now I wonder what an ESB can give me more than what we have.

    If I’m correct ESB relies heavily on XML and open standards, well I can tell you that almost non of our business partners have web-services, and they might know what XML is. That’s why the client connector is a windows service to you only need a windows OS and HTTP (which 95% do have).

    We also wanted to move away from point to point interfacing, but in some regards ESB is going back to that. I’ve seen articles published that said : we don’t even need ESB, the services will connect directly. Now how are you going to manage that. With central hub, you can easily sustain change, in ESB or point to point I don’t see how you can manage that.

    What is see is that I have extended the 2 failover hubs with services that can route and can deliver NOW, which don’t rely on heavy changes required by other apps. Having dynamic routing, transformation, management on the interfaces etc. And it works !

    Now please can someone tell me why I should invest in an ESB, what is lacking?

    Regards,

    Hans.

Skip to main content