The Formal Goodness of Agile Software Architecture – Part 1

  • Article

The purely addictive aspects of working at The Big Show for a geek like me cannot be underestimated. In many ways Microsoft is like any other Fortune 500 company I’ve worked or consulted for, but I’m often reminded that working for Microsoft is different – it’s pure crack for geeks.

As an example, I recently attended a talk given by one of Microsoft’s Distinguished Engineers (DEs) on the subject of Software Architecture. One of the core tenets of this talk was the classification of what constitutes “Architecture” and what constitutes “Design”.

This is a subject I personally have found fascinating for a long time. In my years as an Architect I’ve had to explain the differences between these two highly abstract ideas on a number of occasions. Over time I became convinced that I had some really insightful, if not profound, explanations on the subject.

Later, when I was afforded the singular privilege of working with some very senior Developer talent (BTW, I define “senior Developer” based on knowledge and skill, not necessarily years of experience) in successfully developing software using Agile practices, I started to second-guess the wisdom of my explanations on what constitutes “Architecture” and “Design”.

Over the past few years I haven’t really had a satisfactory way of thinking about these concepts that mapped to my experiences doing Agile “the right way”.

It was during the course of the DE’s talk that I realized that this Architect may have found the “secret sauce” (to steal a term from Gabriel) I was looking for.

This particular DE strongly recommended an article where the authors (Eden and Kazman) formalize the distinction between “Architecture” and “Design”.

What I read was a simple and elegant way to classify what is “Architecture” and what is “Design”.

What I read explained the goodness that I’ve seen in building software in an Agile way.

What I read was pure, unadulterated crack…

 

Intension/Locality Thesis

In the article Eden and Kazman propose a formal basis for the distinction between the concepts of “Architecture”, “Design”, and “Implementation”. This distinction is formalized as the Intension/Locality Thesis:

  1. Architectural specifications are intensional and non-local;
  2. Design specifications are intensional but local;
  3. Implementation specifications are both extensional and local.

For the purposes of this post I’ll be ignoring the third point above, focusing instead on how the Intension/Locality Thesis applies to Agile Software Architecture. Before I can do this, I need to talk a bit about Intension and Locality.

NOTE – The rest of this post uses the term “specification” since the cited article uses this term extensively. However, I personally use this term very loosely. In my mind an Agile specification doesn’t necessarily entail a Word document or UML diagram. A whiteboard session with the Developers where certain decisions are reached (and maybe documented with digital photos), I would argue, also can constitute a specification in the context of this post.

 

What is Intension?

According to the dictionary the relevant definition is:

5. Logic. (of a term) the set of attributes belonging to all and only those things to which the given term is correctly applied; connotation; comprehension. Compare extension (def. 12).

Within the article, the authors further elaborate that specifications are intensional in nature if they define concepts in terms of constraints rather than relying on specific (or extensional) definitions.

For example, the specification “All source code will be written in a .NET language” is intensional in nature. This specification is intensional because it allows any programming language that is compliant with the constraint “a .NET language” (define that how you will ;-) to be used to author source code.

By way of comparison, the specification “All source code will be written in C# and/or VB.NET” is extensional in nature. The extensional constraint explicitly defines which programming languages can be used – C# and VB.NET.

Another way of thinking about intensional specifications is in terms of the number of ways that software can be structured to be in compliance with the specification – for intensional specifications the number of software structures that could be compliant with the spec is infinite (or a very, very large number).

 

What is Locality?

The article formalizes the idea of locality through the concept of subsumption. Essentially, a specification is considered local if every subsequent extension to the software is executed in a manner that complies with the initial specification. That is, every extension to the software subsumes the initial specification.

Since this is a little abstract, another way to think of specifications is that they are local if a subsection of the software satisfies the constraints of the specification without having an effect on the structure of the rest of the software.

By way of comparison, a non-local specification has an effect on the entire software structure – every nook and cranny. In terms of extension, this means that each and every change to the software must be in constant compliance with the original specification.

An example of a non-local specification is the requirement that all .NET classes must eventually have System.Object as a base class. In the case of .NET, this specification is so pervasive that the .NET compiler implicitly adds the System.Object inheritance even if the Developer doesn’t explicitly code it up. Any time you extend existing .NET software with a new class, the extension is always in compliance with this specification.

 

What is Design?

According to the Intension/Locality Thesis “Design” is simultaneously intensional and local. Based on the understandings of intension and locality described above, the following can be deduced about design specifications:

  1. Design provides a specification for software that can be fulfilled in an infinite number of ways.
  2. The constraints imposed by the Design specification are limited in scope – they do not affect every nook and cranny of the software’s structure.

Based on this meaning of Design, the article interestingly points out that the Gang of Four (GoF) arguably provides the quintessential set of Object-Oriented design specifications.

For example, the usage of the Strategy pattern within a set of C# code files fulfills the requirement of being intensional (the Strategy pattern can be coded in an infinite number of ways) and it is also limited in scope (the locality is the set of C# code files).

Additionally, subsequent extensions to the C# code can easily subsume the original design specification. For example, the addition of a Factory for the creation of Concrete Strategy instances to the code subsumes the original design specification. In turn, a further extension to the C# code to incorporate a Use Case Controller that uses the Factory and Strategies subsumes both design specifications, thereby continuing the “chain of locality” for the previous design specifications for the code.

 

What is Architecture?

Like Design, Architecture specifications must be intensional. However, Architecture is non-local – the constraints of Architecture specifications enforce compliance through every nook and cranny of the software’s structure.

The article cites the Law of Demeter as an example of an Architectural specification. From Wikipedia’s article, the Law of Demeter can be written as:

More formally, the Law of Demeter for functions requires that a method M of an object O may only invoke the methods of the following kinds of objects:

  1. O itself
  2. M’s parameters
  3. any objects created/instantiated within M
  4. O’s direct component objects

In reading the above definition it is clear that the Law of Demeter can be implemented in an infinite number of ways (it is intensional). What is also clear is that the Law of Demeter has an effect on every nook and cranny of the software’s structure (OK, this implicitly assumes using the OO paradigm, but you get the point ;-) – it literally dictates the way Developers engineer all of the software (it is non-local).

Another interesting way to think about the Law of Demeter is in terms of subsumption.

Take the hypothetical example of a piece of software that has been engineered in compliance with the Law of Demeter. If this piece of software is extended at a later point in time in a way that isn’t compliant with the Law of Demeter, but doesn’t alter the original structure, then we have the following situation:

  • The new software structure subsumes the original specification
  • The new software structure violates the architecture specification of the Law of Demeter.

Thinking about subsumption gives another way of defining whether a specification is architectural – can extensions to a compliant software structure subsume the original specification and then subsequently violate the original specification? If the answer is yes, then the specification is Architecture, not Design.

The article cites a number of classic examples of architecture specifications that are widespread in the industry, namely Pipes and Filters and Layered Architecture.

 

Agile Software Architectures

What I find so compelling about the concepts described above is that they explain the successes I’ve experienced firsthand in Agile projects. The tools, techniques, and practices that organically grew out of my working with the Developers on these projects can all be easily equated to the Intension/Locality Thesis – we just didn’t know that there was formal goodness that explained why we were successful.

Specifically, I would argue that the Intension/Locality Thesis explains the following in terms of Agile development:

  • What constitutes the Architecture of the software (or the “Metaphor” if you’re into XP)
  • What constitutes the Design of the software
  • What is the purview and concerns of the Architect
  • What is the purview and concerns of the Developer
  • What is the purview and concerns of the Code Reviewer/Code Pair

The first two bullet points are now old hat given the Intension/Locality Thesis, but interpreting the remaining three bullet points in terms of the Thesis and Agile is particularly interesting.

 

The Agile Architect

My experience has been that the Intension/Locality Thesis underlies the ability of an Architect to be agile by formalizing the mechanisms by which the Agile Architect leads through leverage. In terms of the Thesis, this usually takes on two primary forms:

  • The identification of those architectural specifications that most fit the problem space the software is attempting to address. Examples of these architectural specifications include:
    • Layered Architecture
    • Pipes and Filters
    • Law of Demeter
    • Unit test code coverage metrics (e.g., all Use Case Controllers shall have 80+% coverage)
    • Coding standards (e.g., no direct access to class member variables)
  • The identification of design specifications as constraints on the software structure. These identifications transform the design specifications to architectural specifications by making them non-local in nature. Examples of these identifications include:
    • All UI Layer code shall be written using the Model-View Presenter (MVP) pattern
    • All MVP Presenters shall rely on a WCF Service Layer as the Model
    • All WCF services shall delegate implementation to Use Case Controllers
    • The Application Layer Domain Model shall be built as DDD Aggregates
    • All Data Store access shall be implemented using DDD Repositories

You’ll notice a large emphasis on the usage of “off the shelf” architectural and design specifications above (aka Patterns). This is, IMHO, nothing more than the natural manifestation of leverage that organically occurs within successful Agile projects – the result of the need for real-time, high-bandwidth communication between team members. My experience has been that teams inevitably come up with a vocabulary to facilitate high-bandwidth communication. As such, it only makes sense to me not to have to reinvent the wheel every time.

 

The Agile Developer

The Intension/Locality Thesis again provides an elegant formalization for thinking about the role of the Developer on an Agile project. In terms of the Thesis, the Agile Developer is primarily responsible for one thing – engineering software that complies with the architectural specifications.

Now we all know that Agile Developers are responsible for far more than this. For example, Developers are responsible for executing the Agile practices of the team (e.g., CI, Pair Programming, Daily Scrums, etc), but it is interesting to note that on an Agile team the actual engineering constraints are actually pretty light.

However, my experience has been that the best Developers (not to mention the only Developers I would want to work with on an Agile project) self-enforce another engineering constraint on the software – they extensively leverage “off the shelf” design specifications (aka Patterns) in their code.

As an example, take the following hypothetical scenario. A Developer is working on a project where an architectural specification is “All WCF services shall delegate implementation to Use Case Controllers”. As part of the Developer’s Use Case Controller implementation the developer elects to employ the Visitor pattern to apply validation logic on incoming messages to the Controller.

Notice that the Agile Developer doesn’t run her Visitor-based design by the Architect or Dev Lead – they just implement it. No design docs, no design reviews, just the essence of Agile – the creation of quality software by empowered team members.

 

The Agile Code Reviewer/Code Pair

Again, the Intention/Locality Thesis provides excellence guidance as to the activities and concerns of the Agile Code Reviewer (or Code Pair). In line with the Thesis, the Code Reviewer is primarily concerned with the monitoring of, coaching on, and enforcement of the architectural specification constraints within the software.

I would argue that this is a critical idea that often gets lost. If everyone is spending inordinate amounts of time preparing for, conducting, and remediating Code Reviews how Agile can your team actually be? If a team legitimately needs this level of “quality control” then I would argue that the team has likely picked the wrong delivery methodology based on the risk profile of the project (which, BTW, may include the risk that the Developers are not senior enough to do Agile).

I would also add that the best Agile teams have Code Reviewers that naturally taken on a secondary review concern for the identification of any “code smells” within the software. Typically, these code smells tend to take on one of three primary forms:

  • The Developer could benefit from the application of one or more Patterns (design specifications)
  • The Developer applied a Pattern in a suboptimal/non-intuitive way
  • The Developer applied too many Patterns

As I’ve written about previously, the use of Code Reviewers should really be about creating leverage through the federation of architectural stewardship. What the Identity/Locality Thesis gives us is a formal and elegant way to classify exactly what “architectural stewardship” means.

 

Next Time

The Identity/Locality Thesis has got me pretty hopped up right now. As such, I’ve taken a short break from writing about System.Identity and M.

For my next post I’m going to wrap up my thoughts on the Thesis by exploring one last concept – how an Agile Software Architecture can be effectively modeled in UML for easy consumption by the team.

Stay Tuned!