I'm looking at a problem that occurs in different enterprises, from Microsoft to Government, heavy manufacturing to healthcare, distribution to e-commerce, It doesn't occur in every one, certainly not the smallest or the most distributed, but it occurs inevitably as companies grow, and it is in every major corporation.
The Problem is too much overlapping, non integrated, code
In many companies, the job function of IT is basically to write code. This is a problem because sometimes, you shouldn't write code. Sometimes, the problem is best solved by NOT writing code or replacing written code with configured code. (Best solved = lower cost for the enterprise, better features for the customer, quicker turnaround on value).
This is not the result of poor intent. The problem is that we write code when we shouldn't. We write code to get a feature when we should integrate with another app that has a similar feature. We don't use what we have. If it was a 'few incompetent people' causing this problem then the problem wouldn't be widespread. The problem is widespread in every IT group I've worked with in a wide variety of industries and government, both in consulting and as an employee. It is in Microsoft as well.
This problem is systemic. We need a systemic solution. In Microsoft we are solving the problem. My goal is to share this solution with you (and improve it along the way). Of course, to solve a problem, you need to look at the root causes.
From my analysis, I believe that this problem is the result of the lack of visibility, misguided planning, and/or gaps in accountability. I will cover each.
(Call for feedback: After reading the section below, if you feel that there is a "cause" for this problem that I have missed, and which MUST be covered in order to get folks past the barrier of "no one reuses my reusable service," then let me know. I make no claim that my experience represents everyone elses. There are great companies out there. If yours does not have this problem, Look around... tell me if these three elements have been addressed.)
Gaps in accountability
A gap in accountability occurs when no one is responsible for looking for the situations when code should not be written. Your organization can solve this by adding an oversight function SPECIFICALLY chartered to look for situations where integration and service composition was not considered, or not considered important.
In MSIT, this responsibility falls to Enterprise Architecture, but in your organization, it could be in any group that is not chartered to build a project (to avoid conflict of interest). Consider a central PMO or Central Planning group.
Composing a SOBA is a different animal than "adding a feature to app X." It requires forethought and readiness. You have to have a list of appropriate services to draw upon. Even if those services don't (yet) exist, the project that wants to compose them can pay for them to come into existence, but someone still has to create the list.
Without having teams looking for the services that "should" be present, or "could" be present, there is no way to have a list of potential services for a SOBA project to compose from. This is what I call "SOA Planning Governance."
Lack of Visibility
There is a lot going on in a Corporation, and no one wants to spend a lot of money on SOA Planning Governance, so it helps to have the framework in place to handle things like this. A framework means that we can give people guidance on an area that no one has planned yet, on demand.
So when the request comes in to "plan for business need N", we can find the areas of the corporation's IT infrastructure quickly, assign resources with a starting project plan, they follow a well understood process to gather information, produce an analysis, and deliver value. They then go on to other work.
The analysis is used to guide project planning, but the people doing this work are not part of the normal project team. They just do this planning work, over and over. In Microsoft, I could keep four teams busy doing this for five years, easy.
The Solution is an SOA Planning Governance Framework
To address this, I believe that every large organization (facing this problem) needs a SOA Planning Governance Framework. (That's a mouthful. Say that 10 times fast ;-).
Composing the term SOA Planning Governance Framework
- A planning framework is a set of models and processes that conceptually create a "space" where planning can occur. The effort of planning is constrained and guided, which allows it to be completed in a timely manner while producing good results.
- Adding the term "Governance" means that the processes are designed to encourage alignment with business goals, and that misalignments will be captured, reported and visible. The implicit effect is that consequences for alignment will be present as well. Those consequences (good or bad) are outside the scope of the framework but essential to making the system work. People do what you pay them to do.
- A SOA planning governance framework is a framework that helps to plan SOA services when they are strategic and aligned to business goals. It could, theoretically, be used to plan for a DLL or other reusable integration object as well, where it makes sense to create one.
Many of you may have seen me blog about Solution Domain Architecture (SDA). Solution Domain Architecture is a SOA Planning Governance Framework.
Benefits of Solution Domain Architecture
With SDA, you get:
- A well crafted and flexible mechanism for grouping the systems in your organization by features and data (commonality) while factoring out roles and business processes (variability) that should be cross cutting concerns for composable services.
- A process and tools for creating and running a planning team that goes in, solves the problem, and comes out quickly with an analysis that is useful.
- A list of services that may not exist, but should exist, for the enterprise to use. When a project chooses to use a service that doesn't exist yet, then the project gets to pay for the service to come into existence (or a seperate fund can be set up to feed this funding request). The point is that the services can be created "on-demand" and not in advance of an actual customer.
Note that SDA is part of the overall mix, but it doesn't solve every problem. No technique solves every problem. The following tasks are Not solved in Solution Domain Architecture.
- SDA doesn't "design" the services.
- SDA doesn't define what fields go into your canonical schema for 'order' or 'customer'
- SDA doesn't create spanning layers or MDM scenarios.
- SDA doesn't define when you would use ETL, SSB, or ESB to move data.
Solution Domain Architecture helps to create the right list of services for the organization to build. You still need people to design them, build them, and operate them.