Are we ready to prove the “Architecture hypothesis?”

Science progresses on the willingness to take widely held beliefs and test them.  It is one thing to say that a medicine will “cure all that ails ye” but it is another thing altogether to prove that a particular medicine will have a particular effect on health.  Proof is expensive, but science does not march forward without it.

For quite a while, our team has been working diligently to increase the use of architectural models (and architectural thinking) among our IT units.  There are thousands of employees engaged in developing software in MSIT, and it is not always easy to reach a wide audience and make a compelling case.  Our arguments have been based on appeals to logic (clearly, it works), metaphor (it works in other areas of engineering) and anecdotal evidence (it worked on project Foo and see the benefits they got). 

But at the end of the day, we have not been using science.  We never ran a true scientific study to demonstrate that the use of a particular architectural practice improved the outcome of software engineering in a specific and measurable way, compared to a project that did not use architecture.

I’d like to see us, as an industry, get to the point where we can perform a scientific experiment on the efficacy of using software architecture practices to improve software development outcomes.

We’d need a protocol for the experiment, and controlled variables, to reproduce the effects of using architecture.  We’d need a specific definition of what it means to “use architecture” and the specific practices that we believe will have an impact on outcomes.  We’d need a clear way to measure the outcome of the experiment so that, when repeated, anywhere in the world, the experiment could produce comparable results. 

And here’s the rub: I’d like to see it be something that YOU can help with.  The community of architects and developers that care about things like SOA, Architecture, and Planning… we are the people who can perform the experiment in 1,000 settings around the world.   We are the ones that can prove, or disprove, the hypothesis of software architecture.

If anyone has experience with this kind of experiment, I’d love to hear from you, just to see if this idea is too difficult for the community to do.  Personally, I’m not sure where to start.  If you have ideas, please reach out to me.