Unfortunately today, too many solution architects/developers exhibit a common behaviour in the early solution design phases of creating a solution to a proposed business problem - out of frustration. They are not empowered to describe, with the tools they are familiar with, - conceptually - the problem they are trying to solve in sufficient enough terms with increasing levels of refinement. We are just not there yet…
For example, imagine you are an architect/developer just committed to building a new system for your company after gathering, listening and distilling the descriptions of a customer’s business problems, perhaps from several perspectives of those representing the customer. This problem description may for example, result in the deduction that you may be required to construct a solution containing a number of applications and services to satisfy the problem. You already have some of the business requirements, some rules and processes of the system and maybe you have some of the standard solution components and patterns figured out in your head, or bandied around in some email correspondence, or digital notebook, or documentation, guidance etc. and want to begin to start to propose a model of the solution. You want a way to quickly express the problems and possible solutions, as you validate and refine them, taking them from the conceptual to the logical, ready for the transformation to the physical design at some later stage.
Well, hold on, this is a going a little too fast (isn’t it?). Shouldn’t we be writing down formal specifications, documenting the design, or writing source code, building prototypes etc? Well, possibly - if you followed today’s formal or agile practices.
Attempting to start down the track of either of these practices often leads to a frustrating (typically, time-pressured) exercise of trying-out different design tools or unfamiliar techniques, trying to find some tool or process where you can express your emerging design expressions without starting to actually build and validate it or stating it explicitly. In either case, you arrive at an explicit, constrained tool to express your ideas say; using a programming language or other formal description syntax.
“Anyone know a good UML modelling tool? OK, how do I use that one then?”
All too often, the person with the problem and solution ‘in-mind’ (the ‘problem advocate’) gets frustrated with this approach or a particular constrained way of documenting her thinking, and just resorts to opening her favourite source creation tool and trying to describe the problem in detail in all too familiar terms (i.e. source code, XML, etc).
As solution developers, we often use source code because it’s the most generic and flexible medium in which to construct a potentially complex detailed solution to a problem. Since you can document your intent with it, you are also describing the problem you are solving. Its value is its ability to compose smaller components that solve simple problems into a complex solution. At this stage of the design process, we don’t want to constrain our ability to express ourselves in any limited level of detail in the emerging design, so since source code is so generic it is a very powerful tool to do that. However, none of us actually want to spend days or weeks at this stage writing reams of source code to describe conceptual concepts or simplified relationships, especially when the problem description and solution is very volatile in our minds at this stage, and changing designs in source code is extremely arduous and problematic. Especially, if you are a disciplined developer.
But this is what this medium forces us to do. It’s really an all or nothing commitment if you chose this medium, and this often leads to days and weeks of messing around formalising a perfect source description of the current solution and problem 'in mind'. All this while however, the problem ‘in mind’ starts getting polluted by all the constrains of the source code medium (and technologies you chose to model along with it). Inevitably as time ticks on along this path the problem 'in mind' begins to be shaped (warped) by our actual (in)ability to construct a valid solution rapidly to represent the solution at that instant in time. We have moved into the solution creativity stage, with an unfinished exploration of the problem.
Painters, often suffer the same consequence. Once the paint media is dry, they are stuck with it in that form. Consequently, they labour over a mistake, and paint over/around the mistake, or worse, get influenced by the mistake and begin to paint a different picture with the mistake as a feature and lose sight of the original subject.
What we actually need at this stage, apart from 'think time' free from that shackles of constraints, is a higher level malleable description language that allows us to define, refine, drill down in some areas, conceptualise others, state formal requirements, constraints, patterns, components, previous experience, learnings etc. We need to be able to describe this in the natural way that we think, using various media, and human expressions.
Whilst agile programming advocates may state that they don’t suffer this problem, I would disagree that even agile techniques can efficiently capture the intent and design of an idea - as a whole, into related description artefacts as the idea develops. For example, how do you code a whiteboard diagram that could be immediately erased, refined, annotated and tailored from an initial brainstorming session?
This is because of the medium of agile, formal and other programming practices are still to capture the problem in source code along with the solution.
Unfortunately, the ‘source-code-first-approach’ typically leads to further delays and frustration in capturing the problem description and intent or requirements in this medium because it is constantly hampered with side issues and short cuts mostly relating to the source code medium used to describe the problem itself. It takes too long to describe all that information in terms of the language of source code efficiently. This would be fine of course, if you have all this time available perhaps, and of course you could keep the problem untouched, untainted, and innocent 'in mind' all that time.
The real impact of performing this intensive exercise of source code description often removes the focus from the problem definition to the solution implementation, and pretty soon the solution loses sight of the actual problem and becomes tailored to meet the requirements of the solution itself. It becomes slave to its very creation.
The basic problem here is well known; that is: that source code, for example, is a means, a medium (and a very flexible and generic one) for expressing a detailed solution - not a problem.
A New Approach
What if we were given a tool, in a familiar environment, which allowed us to describe the problem using whatever natural expressions we chose? Such as freehand drawings, and have these interpreted as design artefacts?
Of course, this ability of expression could not be completely unconstrained. After all, the task at hand (the domain) is to derive software solutions, so the tools to capture the expressions would have to be constrained in order to be computable.
So, imagine we could pick and choose from a number of different and familiar looking tools that would suit our purpose at the time of describing the problem? These tools could be things like a whiteboard, a scratch pad, a snippet editor, a diagram surface, etc. These tools should not require much user training to get started. They would be intuitive and familiar to our natural ways of expressing our thoughts. They would have to be able to capture information at the speed in which a designer can describe the problem in a format very familiar to them. (For example a design surface such as a ‘whiteboard designer’ would provide always-visible placeholders on the design surface to enter information. Not provide, in some off-to-the-side dialog box, wizard, editor or property grid; editing tools that distract the user from their thought patterns and means to express them.
The tools should understand the author’s expressions (‘problem expressions’) as they add and refine them. Meaning needs to be computed from these problem expressions in terms of a ‘family of possible solutions’. These authoring tools could be anything from free-hand image design surfaces, that recognise simple shapes and concepts for a particular type of design (i.e. system diagram, class diagram etc.), to text editors that capture textual descriptions and specifications, programming artefacts.
These tools would work together (behind the scenes) to guide the author into refining their design. They would converge, consolidate and refactor the expressions and ideas into something that can start to ‘describe’ a general solution domain.
Of course this would require baked in knowledge/experience into these design tools themselves of what particular expressions would mean to a particular problem domain - interpretation. This knowledge/experience and guidance would be refined as the problem description evolves.
Once the problem is sufficiently expressed/defined and consolidated in enough detail to satisfy the tools, the tools could then be asked to make ‘solution suggestions’ or provide guidance on what solution approach to take described in the expressions and related artefacts.
The resulting suggestions/guidance could take many forms. For example, they could provide documentation references, or they could be solution templates to start with, or suggest other tools/technologies/known processes to use. They could be more prescriptive, and offer a selection of known software factories for constructing these solutions. In fact, to take it further with factories, this step could harvest the expressions and artefacts themselves into the instance of the product the factory creates.
It is important to note that it is not until this step of the process (selecting a solution creation tool) that the person defining the problem needs to know of, or understand, these solution construction tools. It would not require them to think in terms of how they define and express their information in technology. This task could be performed by a different role. That role may be familiar with what is available to use for a solution or how to structure it. This is the case today, but unlike today, the solution would not have to assume familiarity of the tools/technologies by the person selecting it.
The problem design tools recommend a set of best guidance/architectures/patterns/designs/tools/processes/technologies/frameworks/factories for the solution instead of only the ones that the problem designer knows of at that time, and the chances of selecting the right tool for creating the solution is greatly increased.
Where are we with this today?
We are currently a long way from seeing solution creation start from a new Visual Studio new ‘Blank Problem’ file. Where, this ‘problem file’ would contain a set a problem expression tools, designers, and guidance that would allow the user to quickly express their problem in loose but structured related terms. These agile tools which allow text, ink, guesture, annotation, etc. that allow further refinement, conceptual prototyping, and eventually, guiding the author down a path to selecting an approach or solution based upon the artefacts and expressions contained in these artefacts – as described above.
Short of providing this ‘New Blank Problem’ file today, the nearest thing we have is to provide them a tailored solution environment (a software factory) that represents a defined specific domain. In other words, we can provide very well defined, integrated design experience tailored to build certain solutions using software factories for example - but this is the last step in the design process.
The missing piece today is being able to give the user the flexibility to think of the problem in very loose terms and being able to capture them and refine them in tools in a ‘workspace’ that recognises the patterns and intent of the design expressions in those tools. Then having those tools interpret those design expressions and offer a selection of tools and factories to use to construct the solution.
Using factories and automation toolkits today requires the author know of the existence and availability of the right software factory to solve the problem at hand, and often this step is pivotal in the success of the solution.