Financial Messaging Services Bus (FMSB) – Add-ons for ESB Toolkit

Summary: Financial Messaging Services Bus (FMSB) is a vertical industry implementation of Microsoft’s Enterprise Service Bus Toolkit 2.0 on top of BizTalk Server 2009 and BizTalk Accelerator for SWIFT. FMSB greatly improves time to market for many complex integration solutions especially in Banking and Capital Markets industries. This paper explains the rationale behind FMSB creation, provides a high‐level description of FMSB architecture, and discusses how FSMB is used to simplify application connectivity to SWIFT. FMSB helps software developers and solution architect by providing components and functionalities within engine which saves development time and give more value from the engine itself.

This document assumes the reader has a basic understanding of generic ESB concepts. For further reading on the Microsoft ESB Toolkit, refer to: https://www.microsoft.com/soa/solutions/esb.aspx Lets use this link: https://msdn.microsoft.com/en-us/biztalk/dd876606.aspx

Financial solutions (but this apply to any industry solution at the end), especially those implementing or leveraging a full-fledged messaging framework , can in fact constitute a foundation platform for the development of a specific application domain (payment or capital markets, government solution, manufacturing,..) where integration technology, data transformation, workflow management and other intermediary services are used to orchestrate transaction flows among different systems running on different, heterogeneous platforms. In addition to messaging, some commonly used services implementing specific behaviors are required for transaction processing, e.g., validation, routing, exception management, and repair. By developing these mechanisms as reusable services, the messaging infrastructure becomes more than an integration framework – it takes on the nature of a bus architecture where the lifecycle of a transaction can be mapped, calling the appropriate services as necessary. This is the essence of the FMSB (ESB): taking common processing services, abstracting and bundling them as reusable services that can be configured at implementation time and track execution KPI data as well as custom defined KPI. In addition, such services can be exposed to third party applications to leverage the preconfigured processing of the bus components, therefore enhancing client value. When defining FMSB, it is important to note that these services are business services, as defined by Microsoft Enterprise Service Bus (ESB) 2.0. The over‐arching concept is to use ESB to orchestrate all services and reuse them as needed.

The basic architectural elements of a financial services application can be categorized into the following segments or layers as shown in Figure 1.

Figure 1: Financial Application Architecture

When considering the architecture of a financial services application, FMSB sits in the layer known as “Business Process and Orchestration”, which is covered by the Microsoft technology stack, and provides integration, orchestration, transformation, and workflow services.

FMSB can be deployed directly to a financial institution client infrastructure project, but also embedded in a Microsoft partner application solution.

FMSB and Microsoft ESB

The FMSB is built principally on BizTalk Server because many of the services are implemented using BizTalk and Accelerator for SWIFT components. To conform to BizTalk ESB architectural best practices, the FMSB was developed upon the BizTalk ESB Toolkit. Most of the FMSB components are very generic and reusable for any solution built on ESB Toolkit.

The base ESB architecture is also the base architecture for FMSB as shown in Figure 2

Figure 2: FMSB and ESB

Financial Messaging Service Bus extends ESB by providing:

• Resolvers which simplify solution creation by implementing support for:
• Multipart messaging (Read Message Part, Replace Message Part)
• Retrieve configuration from Dashboard (FMSB Value)
• Retrieve complex configuration data for SWIFT service (SWIFT Service) (not covered as part of this post)
• Storing itinerary designer value into itinerary runtime
• Loopback adapter (message doesn't leave message box)
• Configuration model for defining BAM tracking data for service/itinerary execution
• Service Broker Orchestration implementation
• Silverlight Dashboard (built on Composite framework (previously known as Prism)) with 5 modules
• Set of Financial services and itineraries together with configuration model (not covered as part of this post)

FMSB Architecture

FMSB Architecture is presented in the following figure:

Figure 3: FMSB architecture as add-on for ESB (red circles represent FMSB add-ons)

FMSB provides:

• Core extensions of ESB (enhanced runtime, tracking KPI during itinerary execution)
• Extended Exception handling (support for invoking pre-defined exception itinerary)
• Loopback adapter (helps to mix messaging and orchestration services inside same itinerary)
• Configuration database (new fmsb resolver extends BRE and UDDI resolvers and provides generic configuration on the Silverlight dashboard)
• Silverlight self-service dashboard (for monitoring Live Data, view KPI Reports, Configuring KPI (BAM), SWIFT service administration)
• Interact and FileAct (specific SWIFT SAG adapters) support (not covered as part of this post)

FMSB Components

FMSB has several modules which could run and be installed separately:

1. CORE modules – those modules could be reused without SWIFT modules. Artifacts include resolvers, adapter, orchestration service broker, database, entity framework models. .
2. SWIFT modules – those modules are connected with BizTalk Accelerator for SWIFT (A4SWIFT) and pre-built for reusing in SWIFT scenario. SWIFT modules use Core modules. Requires A4SWIFT and BizTalk SWIFT adapters installed. (not covered as part of this post)
3. Tracking modules – these module provide enhanced tracking capabilities over ESB and can be installed independently of other modules. They require the BAM infrastructure.
4. Dashboard – modules for presenting Silverlight experience for working with Dashboard capabilities and configuration model. Independent of other modules.

Following Figure present the relation of all FMSB modules.

Figure4: Core + Tracking + Dashboard modules with configuration stores

Need for rich BI within ESB

Like an ocean surrounding an iceberg, business performance management (BPM) provides the business context for performance dashboards, which are layered applications built on a business intelligence and data integration infrastructure (i.e., the base of the iceberg). The most visible elements of a performance dashboard are the scorecard and dashboard screens, which display performance data using leading, lagging, and diagnostic metrics.

In custom implementation extracting relevant business data for Dashboard isn’t an easy task. By using ESB architecture ESB runtime (Dispatcher in messaging scenario and Advance method in Orchestration scenario) has a full control of every message flow inside ESB runtime. But, even with all this knowledge ESB2.0 doesn’t provide full tracking feature. With pure ESB runtime you can’t extract reports which give you answers on common questions:

• How many itineraries/services worked in past ?
• How many itineraries/services currently running ?

In any financial services application, it is very common to provide an answer to questions like the following:

• How many payments have been processed today?
  • How many exceptions did we have?
  • How many were urgent requests?
• How were today’s payments cleared?
  • How many were bulk payments?
  • How many were wire payments?
• Domestic vs cross-border?
  • Who were our top 5 customers today?
  • What percentage of the total came from these 5

These are the main reasons why we enhanced the tracking capability of ESB toolkit.

Tracking architecture

Enriched ESB runtime (with FMSB assemblies „..V1.dll“) now has a capability to extract data by using new BAM interceptor. These data include:

1. Itinerary data (start time, end time, name, version)
2. Service data (start time, end time, business name, status,...)
3. KPI inside Message body (user configure)

Interceptor will extract Itinerary and Service data from Itinerary header. KPI inside Message body would be extracted according to the configuration model and stored into BAM.

Administration of the system would define which itineraries and services should be tracked (ESB Itinerary DSL model), how to extract KPIs from message body, which services/itineraries and define tracking entity (Activity with Checkpoints analogy from BAM). See the screenshot below

With FMSB, configuration of KPIs isn’t done inside Excel (nor custom XML). Administrator of the system (or business person) could use Dashboard with Drag/Drop functionality to define all necessary data (Activities, Checkpoints, Cubes, Measures, Dimensions) together with service position where this data should be tracked.

This model is published in:

1. Configuration model for tracking
2. BAM star schema to persist tracked data.

During the runtime tracking Interceptor would read Configuration model and extract data from Message body as defined.

Dashboard

Dashboard presents visualization of the cubes inside SQL Analysis services. This is generic tool and could be re-used for any cube inside Microsoft SQL Analysis services.

• Sources – Cubes from SQL Analysis services (OrderDocumentSource)
• Measures – Defined measures for selected Cube (CountOf)
• Dimensions – Defined Dimensions for selected Cube (CustomerName, RequestType)
• Filter – Dimension for filtering (same as Dimension).

Dashboard provides several pre-defined type of reports (Column, Line, Pie, Bar, Area, Doughnut, Point, StackedArea) for any source. Following is the sample of Column report:

By selecting different type of report view would be redrawn with the same data.

FMSB installation creates BAM cubes for:

• Storing Itinerary/Services

Those cubes provides source data for report like bellow (percentage of service execution):

• Storing Itinerary/Services as Real Time Aggregation for live service view on the system

LiveData

LiveData view present IT real view on the system. See the screenshot below for details

LiveData presents:

• Current Itineraries working status on the system – „How many itineraries currently working?“
• Current Itineraries status – „Status of itineraries on the system?“
• Current working services status – „How many services currently working ?“

Note: BAM system store data into BAM RTA Cubes with delay.

The above data provides great insight to IT administration to know exactly what the current system/service/itinerary load is.

Important Note: The Tracking architecture extension and the dashboard feature in FMSB are designed to be generic and can be used for any BizTalk ESB toolkit implementation. It is not restricted to financial services vertical. If the BizTalk implementation does not warrant the need for ESB toolkit the dashboard can still be used to view BAM data more visually.

Conclusions

FSMB provides great set of ESB add-ons. With core functionalities benefits are available either for developers or for business persons or for solution architect. By re-using ESB and BizTalk runtime FMSB provides solid foundation for any specific domain development.