The How, the What and the Why - Stages of software development

In the last year or so I have been on a 'mission', a crusade in fact, to complete and promote a software offering that's been under way for the last 3 years. Seems like a long project, but that offering has been evolving and really the journey has been a long evolving cycle for me in my career. As an innovator, I have had to embark on a new phase of precise understanding. I am certainly not focused on the same tasks, nor have the same level of thinking or precise understanding (or vocabulary) now, as I was when I started, or even before I began the journey.

Qualifying how I got to this stage is something I've found in itself very interesting. There have been a few distinct phases in this journey. I am hoping with this article to share some ideas on what these stages are, and how we as software developers grow from developers to designers to architects (dare I say).

A Curiosity

I like to find natural patterns in life for things, and much like my favourite hobby, I've noticed that as you age with something your perspective changes - you mature. I liken this process to growing from a child into an adult and there are definite stages of that growth that everyone goes through, but everyone goes at their own pace, some chose not to go. Most of the when and how is dependent on personal experience, which itself is normally derived from opportunity and an ability to recognise, value and accept change. 

Now, there is a curiosity about how this process manifests itself in different important aspects of a persons life. You could think of the standard 3 major aspects of most adult males lives where they may have encountered this curiosity everyday: domestic life, hobbies and work life.

For example, one of my hobbies. It is an 'extreme sport' (as they still classify it) and so perhaps because of its nature these different behaviours and stages may be exhibiting themselves more clearly than less highly emotional evoking pastimes.

[I am open to being proved wrong about this assertion of course, but this hobby, like many, has a fair few stereotypes, and we spend a lot of time round the camp fire trying to understand what these are, whom they fit and why they fit (and which one we are)]

In this hobby, I'd be considered by most to have more than average level of experience, (time in the sport, a fair number of unique experiences, and a broad range of opportunities), that have forced me to grow up quickly and have a broad view on the subject. I don't claim to be any expert, there are plenty of folk with a lot more experiences than me in many different areas, and to a greater degree in some.

I have applied a number of my own and other's theories about the what the stages of development in that hobby are. I try to test them out on people and discover, some are valid, some not - it is pretty hit-and-miss - and I wonder why that is. Why is it that it's not clearer to me what these stages are and how to quantify them properly? Obviously - I am still stuck at one of these stages myself, which lacks the enlightenment of the others, I need to grow past it and then be able look back at it later. Certainly the more senior in this sport you get the more wisdom you have about this - like all the other aspects of life - of course.

The Stages

However, what still remains clear to me is that there are definite stages of growth/enlightenment, even though I am not sure yet what they are, I know why they exist, and how they came about. What isn't so clear exactly is how to gauge exactly who is at what stage, what to call that stage, and how to precisely define it, what's more, how to distinguish it from other stages.

Now, before we step off the rails and lose the main thread of this, I believe that this process is the same for a hobby as it is for domestic life as it is for professional life. I also assert that I am at a similar 'stage'/'age' in all those aspects of life. In fact, what might be interesting, is that although the timeline scales are different, I am at about the same place in time along each of those paths relatively speaking. Perhaps its linked to our own personal maturity level?

That's assuming that 'time-in-the-saddle', as it were, is the rough measure of your maturity. All things being equal in the world, that may be true, but as we know maturity is not accurately directly related to time, more accurately, it is related to actual experience and (opportunity) and ability to grow from the lessons learned - which is real hard to quantify.

In reality - personally, I find that I am further along the maturity time-line in my hobby than the other aspects of life, and why? well I think this is because I have a greater responsibility to others to share my experiences and help them grow, and I have an environment where people want to learn from those experiences by necessity.

You see, the key for me is this: when you have to explain the how and more importantly the why to someone, the what usually becomes apparent to you (over time). Of course, the better you can articulate the why to someone, so they understand it clearer, the easier it is to understand the what (to yourself). Conversely, once you know the what you can often qualify and quantify the why.

I assert that it's all about self-realisation and the bringing to the forefront of your own mind, that clarifies the what.
I can see a parallel between the stages of learning here, that true understanding of the what is when you surpass the 'conscious-competence' stage, to the 'unconscious-competence' stage, but maintain conscious about the what.

On Software Engineering

Interesting enough maybe, but how the hell does this relate to software engineering?

Well, let's apply these stages to software engineering personas. The How, the Why and the What. Except that we need to make a distinction between the Whatis and the Whatfor. See how you go with this:

Software 'Developers' are all about the how. How best to implement something - usually with the application of technology. They don't necessarily know whatfor they are implementing or why, or even if it has real use or not, they are focused on the best way to do it. The kudos at this stage, coming from peers, is in having the best solution technically, and who can display that the best. The whatis is normally tacit at this stage. The why is accounted for by the hunger for creativity.

The world, to them, at this stage is all very black and white, and most use passion for technical excellence as the tool for reason. It's the 'adolescent' stage of life in many respects.

Now, put that person in front of consumers with real problems and give them purpose and pretty soon they start to understand why they are doing whatfor they do, and why they are creating bits of solutions. For argument's sake, lets call this persona the Software 'Designer'. They start to develop their solution development skills and take a more holistic perspective on their work. The focus is no longer on the best way to technically create something, but the best way to design a solution to solve a given problem. It's the whatfor. The kudos at this stage is drawn from customers who are satisfied by the solution. The how to them is not so important as it was, the whatis is still not that clear, since the customers don't care for either the how or the whatis.

The world to them at this stage is no longer black and white, it's has potential to be very grey anywhere and everywhere. Most use, common sense (from a solution point of view) as the tool for reason. In many respects it is the  'provider' stage in life.

Naturally, some designers chose to remain at this stage by managing programs or products and become the advocate for customers, whilst keeping their hand in the technology. Typically, at this stage, you find comfort in knowing that there are real solutions for real problems and that you have the ability and experience to adapt to provide the right ones.

Others find solace in curiosity and analysing these things and their patterns and purpose in-depth and finding ever greater levels of understanding and precision about whatis these things are exactly that are being defined, selected and applied, and how they differ. At this stage you are trying to find processes and precise meaning as a way to distinguish between things. It's all about using experience and knowledge of experiences to select exactly the right things in the right combinations to create a solution. In order to do that you need to be able to know whatis all the things whatfor they are used for and why. More importantly, when not to use them for any given situation. These are our Software 'Architects'.

At this stage, the world becomes more akin to a series of fine black and white lines (or perhaps a checkerboard), with narrow grey borders. Most use experience, understanding and precision as the tool for reason. This can be likened to the 'wisdom' stage in life.