Supporting Multiple Feature Teams, Multiple Development Cycles and Multiple Releases in an agile way

  • Article

 

In answer to the following question:

“Are you advising that we wait until the build on Main is stable and all dependent systems are ready before we attempt to deploy anything?  This makes a hard-to-hit-target nearly impossible to hit (we have multiple teams, so it is already complex as it is).  We might not ever get anything out the door.  Our stakeholders would prefer a pared-down release on regular intervals versus a deferred release.  Do you plan to address "advanced web development" in your guide?”

No, I am certainly not advising holding up deploying *anything* until *everything* is ready. That is hardly agile, and I am a strong proponent of agile. I agree with your stakeholders that often regular incremental releases are much better than deferred *big bang* releases. Many lives ago, I worked on mainframe applications in the Energy Utility (Gas and Electric) industry. My primary focus was architecting, designing, and building large-scale mainframe Customer Service Systems to support such old-fashioned concepts as Meter Reading (using humans), Cycle Billing, Estimated Billing, Cancel Re-billing, etc.

These systems, back in those days (pre de-regulation) often took years to envision, design, build, stabilize and deploy. Several problems happened with those old *waterfall* approaches, where all of the requirements needed to be gathered up front (takes six months), the architecture and design must be completed next (takes another six months), the code must be completed (takes a year) and then tested (takes forever). All sequentially in huge chunks of phases before the system could be deployed in one huge *big bang* release. These companies would undertake a new CSS development only once every fifteen years (I am not generalizing, I have seen studies of the Energy utility industry).

Often the business had evolving requirements. But because the software development life cycle was waterfall and drawn out over two or three years, the customer was unable to get their evolving requirements incorporated into the new released system. What do you tell them? Wait for the next release in fifteen years? How do you get a business to prioritize their requirements, when they KNOW that anything that is not top priority gets cut and they have to wait fifteen years for priority two requirements? But I digress.

In theory, I think I could propose a branching structure (and process) - a branching strategy that would support your requirements. It does not NEED to be exclusively for *advanced web* development, but would, I suggest support any agile development/release process that has similar characteristics:

  • At any time there may be one, two, or more *released* versions of your product (application, system) which you need to support simultaneously. In your case you seem to have only one *released* version (be it Release 1.0, or Release 2.0, or Release 2.1). You only have one *current* version that you need to deal with (unless I am misunderstanding something). For consistency sake let's call the current released version vCurrent. If vCurrent is Release 2.0 then Release 1.0 might be vCurrent -1, and the next release being stabilized is vNext.
  • At any time there may be one, two, or more feature teams developing new features. Any given feature team is working on a set of one or more features for a future release. The release may be vNext, or vNext+1 or vNext+2.
  • Each Feature team will have one (or more) feature branches to enable the team to isolate their changes from changes made by other feature teams.
  • Before a feature team can merge their code into the Main branch, it has to be stabilized *first* in the feature team branch, it has to pass QA quality gates, and it has to be *ready to release* or *ready to share*. 
  • When the code is *Ready to Release*, that means that the code will become part of the vNext release, and can be merged into the Main branch for stabilization and release.
  • There may or may not be other feature teams. If there are multiple feature teams, they may or may not be working on the same release (vNext). By definition if you have a situation where one team is *Ready for Release* and another team is not, then you are working on development for vNext (Ready for Release) and vNext + 1 or even vNext + n (NOT ready for Release yet).
  • At any time that you want to Release a new major or minor version, you select which feature teams are *Ready for Release* and which are not. You merge, one at a time, the latest version of the feature branch for each of the feature teams that will be part of the vNext release.
  • If a team is *Not Ready for Release* then you do NOT merge their code into the Main branch since they are not ready.  They will have their features merged into Main only at such time as their feature is ready for release and is determined to be part of the vNext release (from Main).

So all of this boils down to a few key questions:

  • How many releases do you need to support? I assume vCurrent, but also vCurrent -1? or vCurrent - 2.
  • Do you issue hotfixes to any of the supported releases?
  • Do you issue Service Packs to any of the supported releases?
  • How many development (feature teams) do you have?
  • Are these feature development teams all working on the next release (vNext) or are thy working on multiple releases in parallel (vNext, vNext +1, vNext +2)
  • How often do you do a major release?
  • How often do you do a minor release?

Finally the approach:

Think of the development side of the branching strategy separately from the release side:

  • All releases are stabilized in Main and branched when shipped to one (or more) branches on the *release* side
  • All releases are developed in one or more branches on the *development* side.
  • Bugs that are identified during stabilization of a release (but before the release is shipped) are fixed on the development side
  • Bugs that are identified after a release ships are fixed on the release side.
  • You may or not need a read-only branch for safe keeping
  • You may or may not need safekeeping branches for each hotfix or service pack
  • If you do NOT issue hotfixes or service packs, consider the basic branch plan (on the release side)
  • If you issue hot fixes (but not service packs) consider the standard branch plan (on the release side)
  • If you issue both hot fixes and Service packs to a released version, consider the advanced plan (on the release side)
  • If you have only one version under development (vNext), and only one feature team, you have a simple branching structure on the *development* side (Main + Development)
  • If you need feature team isolation (either for teams working on the same release, or for teams working in parallel on multiple future releases - add complexity to the *development* side - by adding more feature branches.
  • Unless you can make a strong case for why you need an extra *isolation* branch layer, between all (or some) of the feature branches and the main branch, do not add it.
  • Integrate features (or sets of features from a single feature branch) into main - one at a time.
  • After integrating a feature branch into Main, integrate Main into all of the other feature branches - daily
  • Keep main as stable as possible - it is used for QA stabilization of vNext and should not have vNext +1 or vNext +2 code until vNext has shipped.
  • Wherever possible, branch and merge from the latest version of a branch rather than doing cherry picking
  • Consider an integration branch layer between Main and the Feature branches:
    • When you are working on multiple future releases in parallel (vNext, vNext + 1, vNext +2) and for any version you have multiple feature teams working in parallel that may need to share or integrate their work. So for example, while Main is used to stabilize vNext release, you might have an integration branch (between Main and the vNext+1 branches) so that the vNext +1 can start doing integration testing while Main is still dedicated for vNext stabilization
    • When you have multiple feature teams working on the same release - and you want to do integration testing and sharing code between teams, but you do not want to destabilize the Main branch and you do not want to interfere with forward progress in the feature teams while integration is being tested (code velocity). Consider automated testing of builds in the development branches as a preferred way to improve code velocity.
  • Code (for development and bug fixing during development) should be checked into development branches (or feature branches) and tested before merging with Main
  • Code should not be checked into Main directly. Code can be merged into main once; tested.
  • Code for post release bug fixing is checked into the servicing branch (service pack or hotfix) for a release. But the release branch is locked down when released (shipped) and made read-only

Thoughts?