Placing Architecture Properly into Scrum Processes

As I’m about to complete my share of a longer engagement on using Lean principles to improve the processes at an online services firm, it occurred to me that the efforts we undertook to properly embed Architecture practices into their Scrum process were novel.  I haven’t seen much written about how to do this in practice, and I imagine others may benefit from understanding the key connection points as well.  Hence this post.

First off, let me be clear: Agile software development practices are not at all averse to software architecture.  But let’s be clear about what I mean by software architecture.  In an agile team, most decisions are left to the team itself.  The team has a fairly short period of time to add a very narrow feature (described as a user story) to a working base of code and demonstrate that the story works.  The notion of taking a couple of months and detailing out a document full of diagrams that explains the architecture of the system: pretty silly. 

The value of software architecture is that key decisions are made about the core infrastructure of the system itself: where will generalizations lie?  Will a layered paradigm be used, and if so, what are the responsibilities of each layer?  What modules will exist in each layer and why will they be created? How will the responsibilities of the system be divided up among the layers and components?  How will the modules be deployed at scale?  How will information flow among the modules and between the system and those around it?

The way these questions are answered will indicate what the architecture of the system is.  There are many choices here, and the “correctness” of any choice is a balance between competing demands: simplicity, security, cost, flexibility, availability, reliability, usability, correctness, and many more.  (These are called “System Quality Attributes”).  Balancing between the system quality attributes takes thought and careful planning.

So when does this happen in an agile process?

Let’s consider the architect’s thinking process a little.  In fact, let’s break the software architecture process into layers, so that we can divide up the architectural responsibility a little.  You have three layers of software architectural accountabilities.  (Repeat: I’m talking about Software Architecture, not Enterprise Architecture.  Please don’t be confused.  Nothing in this post is specific to the practice of Enterprise Architecture).  All this is illustrated in the diagram below.  (Click on the diagram to get something a little more readable. 

At the top, you have the Aligning processes of software architecture.  These processes consider the higher levels of enterprise architecture (specifically the business and information architecture) to create To-Be Software Models of the entire (or relevant) software ecosystem.  If you’ve ever seen a wall chart illustrating two dozen or more software systems with connectors illustrating things like flow of data or dependencies, you’ve seen a model of the type I’m referring to.  Creating and updating these kinds of diagrams is a quarterly or semi-annual process and reflects the gradual changes in the strategy of the enterprise. 

In the middle, you have the Balancing processes of software architecture.  These processes consider the needs of a single system but only from the level of deciding why the software will be divided up into modules, layers, and components, how that division of responsibility will occur, and what the resulting system will look like when deployed in specific technologies in a specific environment.  All of this can be conveyed in a fairly short document that is rich in diagrams with a small amount of text explaining the choices.  This occurs once when a system is moving forward, and the software architecture can be developed alongside the first couple of sprints as input to the third and subsequent sprints.

At the bottom, you have the Realization processes of software architecture.  This is where the architecture becomes software, and this is where decisions are made about the choice of specific design patterns, the appropriate level of configurability vs. simplicity, and the ability to demonstrate whether the actual intent of the architecture occurs in practice.  In agile, this layer occurs within the team itself.  The software architect can offer advice about what patterns to use, but it is up to the team to realize that advice and/or decide not to implement it.  The team will very likely implement the software architecture as described, but may choose to improve upon it.

What does the process look like

There are many visualizations of scrum running around.  Some are described in books, others in papers or blog posts.  Most share some common elements.  There is a product backlog that, through the magic of sprint planning, the team extracts a subset for the sprint.  This becomes the sprint backlog.  The illustrations then tend to show the various rhythms of Sprint as cycles (sprint cycles and daily cycles), ending with a demonstration and retrospective.

In order to illustrate and train a team on all elements, including the business analysis elements, we chose to be a bit more explicit about the steps PRIOR to sprint planning, including the processes of creating and improving the stories prior to the start of a sprint.  (as above, click on the image to enlarge it).

Astute observers will notice that we added a step that we are calling “pre-sprint story review.”  This is a meeting that occurs one week prior to the start of a sprint.  It is called by the product owner and he or she invites “senior chickens” (architects, user experience leads, development and test leads, and any other “non-team” members that want a say in the sprint. 

In that week prior to sprint planning, those folks, working with the product owner, can improve the stories, add constraints, refine the description and acceptance criteria.  And here’s where the architects get to play.  Architects fulfilling the role of “Balancing” in the model above will have (or can create) an outline document describing the architecture of the software system, and can “link” that document to the SPECIFIC STORIES that are impacted by that design. 

(Note: assuming you are using a tool like Microsoft’s Team Foundation Server, that fully enables Scrum in multiple forms, this is a nearly trivial activity since a document can be easily linked to any story.  Enough advertising.)

So is an architect a chicken or a pig?  Answer: depends on what “layer” the architecture is at.  The top two layers are chickens.  The third layer, realization, is done by the team itself.  The person on the team may or may not have the title of “designer".  (I’d prefer that they did not, but that’s just because I believe that ALL team members should be trained to fulfill that role.  In reality, the skill may not be wide spread among team members).  Therefore, the third layer is done by the pigs. 

I hope this provides some insight into how a team can embed software architecture into their scrum cycles.  As always, I’m interested in any feedback you may wish to share.