First of all, let’s level set on the definition of system Flexibility. I quite like M. Barbacci’s definition which is “Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.”


The beauty of this system quality attribute is that it is a key factor in providing Agility (“The ability of a system to be both flexible and undergo change rapidly” –MIT). In fact, I’ll blog some ideas I’ve been thinking as a formula for Agility because I feel Flexibility doesn’t seem to get the credit it deserves here…anyway, for arguments sake, let’s just assume that Flexibility is an important system quality attribute.


The problem I’d like to raise for discussion is how to help system designers build systems that are optimized for Flexibility. One approach I’ve been thinking about to help system designers build systems that are optimized for system Flexibility is to create an Architecture Principle which includes direction and explanation so that it can easily be consumed by system designers. I’d like to throw one out to this Architect Community to see what you all think.


The approach I’ve taken is


1.      Identify well-known patterns that optimize for Flexibility in high and low level system abstraction levels.


2.      Identify common scenarios in my IT environment and use them to explain what to do in those scenarios to optimize for Flexibility


3.      Identify Flexibility anti-patterns, or patterns which degrade system Flexibility.


Using this approach this, I’ve created a document which contains the content of this design principle. Below is a first go at this principle:


Design for Flexibility


Description: Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.


 


Rationale:


Optimizing for system Agility is a primary goal for enterprise systems with the purpose of enabling rapid delivery of enterprise applications to the business. There are several system quality attributes which are used to calculate system Agility such as Reusability, Interoperability and Flexibility. Within Microsoft IT’s enterprise application environment, system Flexibility is of particular importance as it is a key factor contributing to an agile architecture. For this reason, attention is paid to ensure guidance is provided to system designers to ensure systems are optimized for Flexibility.


 


Implications:


If systems are designed to optimize system Flexibility, the expected benefits are:


·         Enabling business process changes with minimum effect on technology platforms.


·         Enabling technology platform changes with minimum effect on business processes.


·         Enabling Service Oriented Architecture (SOA).


·         Enabling modularized systems library for rapid system reuse.


·         Improving system Maintainability.


A note about when and where to optimize for Flexibility: This sections includes a few scenarios commonly observed in Microsoft IT to help guide system designers where and how to optimize for Flexibility.


1.    System Integration. There are three primary Interface Types to send and receive data between systems and they are; Real-time Integration, Near Real-time Integration and Batch Integration. Each of these scenarios are explained in the sections below.


a.    Real-time Integration


                                          i.    The table below describes the scenario when Real-time Integration is often the best choice:























Term


Value


Transaction Volume – Traffic


Greater than 1 Peak Transactions Per Second


Transaction Volume – Message Size


Less than 10Mb Peak Message Size per Transaction


Interface Response Time


Less than 5 seconds


Latency Tolerance


Less than 5 seconds


Data State Staleness


Less than 60 seconds


                                         ii.    The table below lists patterns and styles which optimize systems for Flexibility for Real-time Integration scenarios. See Appendix A – Pattern Reference for details on these patterns.















































Name


Adapter


Chain of Responsibility


Command


Data Transfer Object


Entity Aggregation


Event Message


Event-Driven Consumer


Façade


Functional Integration


Gateway


Message Broker


Message Bus


Message Endpoint


Messaging


Messaging Gateway


Observer


Pub/Sub


Remote Façade


Service Layer


Service-oriented Integration


 


b.    Near Real-time Integration


                                          i.    The table below describes the scenario when Real-time ETL is often the best choice:























Term


Value


Transaction Volume – Traffic


Less than 1 Transactions Per Second at Peak period and greater than 1 Transaction per 30 minutes at Trough period


Transaction Volume – Message Size


Greater than 10Mb and less than 30 Mb Peak Message Size per Transaction


Interface Response Time


Greater than 5 seconds and less than 3 minutes


Latency Tolerance


Greater than 5 seconds and less than 30 minutes


Data State Staleness


Less than 30 minutes


                                          i.        The table below lists patterns and styles which optimize systems for Flexibility in Near Real-time Integration scenarios. See Appendix A – Pattern Reference for details on these patterns.















































Name


Adapter


Chain of Responsibility


Command


Data Transfer Object


Entity Aggregation


Event Message


Event-Driven Consumer


Façade


Functional Integration


Gateway


Message Broker


Message Bus


Message Endpoint


Messaging


Messaging Gateway


Observer


Pub/Sub


Remote Façade


Service Layer


Service-oriented Integration


 


c.    Batch Integration (aka ETL)


                                          i.    The table below describes the scenario when Batch Integration via ETL is often the best choice:























Term


Value


Transaction Volume – Traffic


Less than 10 Transactions per day at Peak period


Transaction Volume – Message Size


Greater than 30 Mb Peak Message Size per Transaction


Interface Response Time


Greater than 3 minutes


Latency Tolerance


Greater than 30 minutes


Data State Staleness


Greater than 30 minutes


                                         ii.    To optimize Flexibility in Batch Integration scenarios,


1.     Use the following Object-oriented Design patterns; Façade and Adapter even within SQL Stored Procedures. See Appendix A – Pattern Reference for details on these patterns.











Name


Adapter


Façade


2.    Follow guidelines to implement the concepts of encapsulation and ‘separation of concerns’.


d.    SAP Web Services. This scenario is when an application requires integration to an SAP BAPI which is wrapped with a Web Service application. To optimize for Flexibility, the Web Services which directly wraps the SAP BAPI should implement the following Object-oriented Design patterns; Façade, Canonical Data Model and Adapter. See Appendix A – Pattern Reference for details on these patterns.


2.    Configuration Management scenarios. Systems which have been designed to optimize for Flexibility take advantage of configuration mechanisms allowing for changes to the behavior of an application via changes in a system configuration file.


a.    Types of Configuration Management changes that optimize for Flexibility are


                                          i.    Configurable Logging sources and log message types


                                         ii.    Configurable modules for in-process interception – Note that the Chain of Command Object-oriented Design pattern should be used in these situations.


 


 


Measure:


·         é Green  The use of Architecture patterns which optimize Flexibility are used at all architecture and design abstraction levels as well as an implementation of Configuration Management for all major non-functional and functional sub-systems and system components.


o    Examples of proven Architecture Styles and Architecture and Design patterns which optimize system Flexibility are in the table below. See Appendix A – Pattern Reference for details on these patterns.















































Name


Adapter


Chain of Responsibility


Command


Data Transfer Object


Entity Aggregation


Event Message


Event-Driven Consumer


Façade


Functional Integration


Gateway


Message Broker


Message Bus


Message Endpoint


Messaging


Messaging Gateway


Observer


Pub/Sub


Remote Façade


Service Layer


Service-oriented Integration


 


o    Types of Configuration Management changes that optimize for Flexibility are


§  Configurable Logging sources and log message types


§  Configurable modules for use process interception – Note that the Chain of Command Object-oriented Design pattern should be used in these situations.


Separation


 


·         è Yellow – Architecture patterns which optimize Flexibility at the design abstraction level. Examples of such design-level patterns which optimize system Flexibility are:





























Name


Adapter


Chain of Responsibility


Command


Data Transfer Object


Entity Aggregation


Event Message


Event-Driven Consumer


Façade


Message Endpoint


Observer


Remote Façade


Note: See Appendix A – Pattern Reference for details on these patterns.


 


·         ê Red – System implementations which:


o    Do not implement any of the styles and patterns listed in Appendix A – Pattern Reference with a Direct or Indirect value in the “Relevance to Flexibility” column.


o    Are using patterns such as


§  Shared Database


§  Remote Procedure Invocation


o    Do not follow guidelines for Encapsulation and Separation of Concerns


o    Do not have configurable sub-systems or application components.


o   Implement a Batch Integration system integration type for a Real-time or Near real-time scenario


1.1 Appendix A – Pattern Reference
























































































































Name


Relevance to Flexibility


Type


Reference


Remote Procedure Invocation


Anti-pattern


Style


http://www.enterpriseintegrationpatterns.com/EncapsulatedSynchronousIntegration.html


Shared Database


Anti-pattern


Style


http://www.enterpriseintegrationpatterns.com/SharedDataBaseIntegration.html


Adapter


Direct


Design


http://www.dofactory.com/Patterns/PatternAdapter.aspx


Event Message


Direct


Design


http://www.enterpriseintegrationpatterns.com/EventMessage.html


Façade


Direct


Design


http://www.dofactory.com/Patterns/PatternFacade.aspx


Functional Integration


Direct


Architecture


http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archfunctionalintegration.asp


Messaging


Direct


Style


http://www.enterpriseintegrationpatterns.com/Messaging.html


Service Layer


Direct


Architecture


http://www.martinfowler.com/eaaCatalog/serviceLayer.html


Service-oriented Integration


Direct


Architecture


http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archserviceorientedintegration.asp


Chain of Responsibility


Indirect


Design


http://www.dofactory.com/Patterns/PatternChain.aspx


Command


Indirect


Design


http://www.dofactory.com/Patterns/PatternCommand.aspx


Data Transfer Object


Indirect


Design


http://www.martinfowler.com/eaaCatalog/dataTransferObject.html


Entity Aggregation


Indirect


Architecture


http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archentityaggregation.asp


Event-Driven Consumer


Indirect


Architecture


http://www.enterpriseintegrationpatterns.com/EventDrivenConsumer.html


Gateway


Indirect


Architecture


¡ http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/desgateway.asp


¡ http://www.martinfowler.com/eaaCatalog/gateway.html


Message Broker


Indirect


Architecture


¡ http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebroker.asp


¡ http://www.enterpriseintegrationpatterns.com/MessageBroker.html


Message Bus


Indirect


Architecture


http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebus.asp


Message Endpoint


Indirect


Architecture


http://www.enterpriseintegrationpatterns.com/MessageEndpoint.html


Messaging Gateway


Indirect


Architecture


http://www.enterpriseintegrationpatterns.com/MessagingGateway.html


Observer


Indirect


Design


¡ http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpatterns/html/DesObserver.asp


¡ http://www.dofactory.com/Patterns/PatternObserver.aspx


Pub/Sub


Indirect


Architecture


¡ http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/despublishsubscribe.asp


¡ http://www.enterpriseintegrationpatterns.com/PublishSubscribeChannel.html


Remote Façade


Indirect


Design


http://www.martinfowler.com/eaaCatalog/remoteFacade.html


 


 



Appendix A Glossary














































































































Term


Definition


Agility


Ability of a system to be both flexible and undergo change rapidly.


Architecture Pattern


An architectural pattern expresses a fundamental structural organization schema for software systems. It provides a set of predefined subsystems, specifies their responsibilities, and includes rules and guidelines for organizing the relationships between them.


Architecture Style


An architectural approach to solving a common problem.


Business Capability


The unit describing the combination of people, business process and technology and the policy around its management


Component


Defined as executable/source code, practices, patterns, services as prescribed by Technical Architecture team.


Data State Staleness


The difference between the master state, from which the cached state was created, and the current version of the cached state.


Design Pattern


A design pattern provides a scheme for refining the subsystems or components of a software system, or the relationships between them. It describes a commonly recurring structure of communicating components that solves a general design problem within a particular context.


Encapsulation


Encapsulation conceals the exact details of how a particular class works from objects that use its code or send messages to it. So, for example, the Dogs class has a bark() method. The code for the bark() method defines exactly how a bark happens (e.g., by inhaling() and then exhaling(), at a particular pitch and volume). Timmy, Lassie’s friend, however, does not need to know exactly how she barks. Encapsulation is achieved by specifying which classes may use the members of an object. The result is that each object exposes to any class a certain interface — those members accessible to that class. For example, an interface can ensure that puppies can only be added to an object of the class Dog by code in that class. Members are often specified as public, protected and private, determining whether they are available to all classes, sub-classes or only the defining class. Some languages go further: Java uses the protected keyword to restrict access also to classes in the same package, C# and VB.NET reserve some members to classes in the same assembly using keywords internal (C#) or Friend (VB.NET), and Eiffel allows one to specify which classes may access any member


Enterprise Solution


Any system that participates in an end to end business process that spans beyond the organization. The solution is SAP, Siebel, Clarify,/Metropolis, *ebis, or a solution that masters a key data entity such as Organization, Partner, Customer, Solution Opportunity, Campaign, Product or Order (among others).


Extract Transform and Load (ETL)


Extract Transform and Load is an approach to create nearly-identical copies of the data and to manage the integrity of the copies if they can be updated at both the source and target within a replication interval.


·         http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpatterns/html/ArcDataReplication.asp


Flexibility


Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.


Implementation Pattern


An implementation pattern is a low-level pattern specific to a particular platform. An implementation pattern describes how to implement particular aspects of components or the relationships between them, using the features of a given platform.


Intentional Integration


Defining a public data and service contract to the process and data owned by a system. Should not be consumer driven decision


Interface Response Time


Amount of time a consumer application is expected to wait for a response from the provider application.


Interoperability


the ability of two or more systems or components to exchange information and to use the information that has been exchanged.


Latency Tolerance


Delay between updates between the Source Application and the Destination Application. Some forms of data integration imply a delay between updates to the data that is used by multiple applications. For example, in the Data Replication pattern, the data is extracted from the source system, and it is transported over a network. The data might then be modified, and then it is inserted in a target database. This delay means that one system may have access to data that is more up to date than another system


Maintainability


Maintainability is:


·         The aptitude of a system to undergo repair and evolution.


·         The ease with which a software system or component can be modified to correct faults, improve performance or other attributes, or adapt to a changed environment. (2) The ease with which a hardware system or component can be retained in, or restored to, a state in which it can perform its required functions.


Near Real-time Integration


General term for describing asynchronous inter-system communication typically an implementation of the Message Bus Integration Style.


·         http://www.enterpriseintegrationpatterns.com/ComposedMessagingMSMQ.html


·         http://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebus.asp


Performance


Performance is the responsiveness of the system – the time required to respond to stimuli (events) or the number of events processed in some interval of time.  Performance qualities are often expressed by the number of transactions per unit time or by the amount of time it takes to complete a transaction with the system.


Process


A process is the business process that an application may be involved in executing or automating


Real-time Integration


General term for describing synchronous inter-system communication typically an implementation of the Request-Reply pattern. Note that Real-Time Integration can also implement asynchronous communication but in a fashion to scale synchronous communication.


Relevance to Flexibility


§  Direct. The pattern/style directly contributes to optimizing system Flexibility.


§  Indirect. The pattern/style indirectly contributes to optimizing system Flexibility.


§  Anti-pattern. The use of the pattern adversely affects system Flexibility.


Reliability


Reliability is the ability of the system to keep operating over time.  Reliability is usually measured by mean time to failure.


Reusability


Reusability is the degree to which a software module or other work product can be used in more than one computing program or software system.


This is typically in the form reusing software that is a encapsulated unit of functionality.


Role


A collection of responsibilities and motivations, to which authorizations are attached.


Scalability


Scalability is the ability to maintain or improve performance while system demand increases.


Security


Security is a measure of the system’s ability to resist unauthorized attempts at usage and denial of service while still providing its services to legitimate users.  Security is categorized in terms of the types of threats that might be made to the system.


Separation of Concerns


Separation of concerns is the process of breaking a program into distinct features that overlap in functionality as little as possible. A concern is any piece of interest or focus in a program. Typically, concerns are synonymous with features or behaviors


Supportability


Supportability is the ease with which a software system is operationally maintained. 


Testability


Testability is the degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met.


Transaction Volume – Peak Message Size


The largest message size expressed in bytes for a single transaction.


Transaction Volume – Peak Traffic


The largest number of transactions expected to be processed expressed in ‘transactions per second’.


Type


§  Architecture. See Architecture Pattern.


§  Design. See Design Pattern.


§  Style. See Architecture Style.


Usability


Usability is:


§  The measure of a user’s ability to utilize a system effectively.


§  The ease with which a user can learn to operate, prepare inputs for, and interpret outputs of a system or component.


§  A measure of how well users can take advantage of some system functionality.  Usability is different from utility, a measure of whether that functionality does what is needed.


 


What do you think?


How do you provide Architecture/Design guidance to optimize for system Flexibility?