Efficiency vs. Productivity in Software Development
Regardless of the methodology used, software development has a mission: delivering value to customers through software products and services, which serves the ultimate goal, making money!
One of the benefits that formal processes like Waterfall were supposed to deliver was to eliminate the chaos through the development of large projects, i.e. making the development more efficient.
However, I believe these models went too far and made the development “over efficient”. All the far ahead planning, allocating resources, and trying to stick with the schedule preempted all buffers in the development team’s capacity and left no room for reacting to the ever changing environment and dynamisms of the software development. The result is a reduction in the output or the value delivered to the customer. What agile practices have been emphasizing on is “productivity matters the most”.
Let me draw an analogy to make the matter more clear:
Consider a factory that manufactures a product. To do that, this factory utilizes four different machines M1, M2, M3, and M4. The raw material is the input to M1 and M2. The M1’s output goes into M3, and the output of M3 and M2 flows into M4, which outputs the final product. Also assume that the M3 is a bottleneck, that is, its capacity is less than that of the other machines.
To reach the maximum efficiency for each machine (based on the definition of efficiency from Mariam-Webster: effective operation as measured by a comparison of production with cost, as in energy, time, and money), we want them to be working 24/7. We invested a lot in these machines and the more idle they are, the less their individual efficiency will be. Now let’s look at the big picture: all machines are producing with their maximum capacity. Since M3 is a bottleneck, there is a pile of M1 parts in front of it ready to be processes. M4 also has a lot of M2 parts ready to go, but it is starving on M3 parts.
There are many major issues with this system, a few that comes to mind are:
1) It only can produce at the same pace as that of its bottleneck,
2) A lot of inventory is sitting to be processes (consider the risk of damage, cost of storage, etc.),
3) What if there is a required maintenance time for M3? Since we do not want to reduce the efficiency of other machines, we do not bother to pay the cost of repurposing another machine to produce M3 part, which is on the critical path of the product.
4) Any change in the product is very costly. Let’s say we do need to change the design of M1 part; all the excess M1 inventories are now wasted.
Now let’s redesign the system a bit: maybe M2 can be used to produce M3 parts as well (increasing the throughput), but there needs to be a setup cost to switch between parts each time. Also we slow down M1 to produce only as much as what machines that produce M3 parts can process (adding some idle time to M1). As you see, we decreased the efficiency of the individual machines to achieve higher productivity. We now produce more than before, deliver more to our customers and make more money.
A software development team is just like that factory: it produces products using limited resources with various capacities, only add the uncertainty of how long an activity would take and the fact that the 'machines' are much more sophisticated and multi-purpose. You can relate all the items I mentioned for the factory to a software development team. If the team shares the same goal and on the same mission I mentioned above, it needs to focus on its productivity, not efficiency! Any process or activity that the team goes through should get the team closer to its final goal. Conforming to this mindset is not easy though, it requires a culture change.
Once I heard from a project manager that “we pay these people, so better keep them busy all the time”! That is exactly the kind of thinking that could blow up a team’s productivity.
Final word: I believe the most efficient team is not the most productive one!
--
Good day to you
Hamed