Agile Methodology and New Product Development: Four Key Issues
Alok Srivastava, Solution Architect ISV Practice, Microsoft Corporation
During all my years in the software Industry, I have architected, designed, developed and managed many applications. Many of these projects started from concepts and many were simply addition to an existing system. The only thing I found common in all these projects is that they were all different and required different approaches to architecture, design and development to finish on time and under budget. Recently I have seen a surge into adopting “Agile” methodology for developing software. This is a very positive shift and it is paving the way for development teams to deal with shorter development cycles and it is enabling them to release features faster rather than waiting for multi-year release cycles. Overall, the benefits of agile development outweigh any additional efforts that may be needed to make it work.
Agile methodology, however, requires a different kind of focus. Before entering into agile development, it is important that development teams have a good understanding of what needs to be in place for them to successfully adopt this methodology. Feature addition to an existing product or building a brand new product, SaaS delivery, on-site deployment or shrink-wrapped product all have impact on the pre-condition that must be met to make agile development painless and effective. Over time I have seen many projects get delayed or not gain any benefit simply because they started using agile methodology prematurely.
In this Blog I would like to focus on the challenges of using agile methodology for developing a new product. Other scenarios will be discussed in my future Blogs on this topic.
Agile Methodology and New Product Development: Taking a new product idea and turning it into a delivered product is quite challenging and it has lots of unknowns to deal with. Using agile methodology for these projects is desirable but very challenging due to a number of risk factors. Most common risks that are likely to impact a new product development using agile methodology are:
1. Changing core requirements
2. Unknown technical challenges that have not been verified with a prototype
3. Incomplete system architecture
4. Lack of framework for continuous integration
All of these can prove to be the stumbling blocks that can derail the project and make the agile development totally inefficient. Let’s take a quick look at the impact of each of these:
1) Changing Core Requirements: Agile methodology works very effectively in managing changes to feature level requirements as long as the core architecture can stay stable. Any requirement changes that impact the core of the system or require changes in the architecture will impact agile development adversely. In a new product, this is quite common and many times the time pressure forces teams to jump into development without waiting for the core to stabilize. During this phase, some of the development work can make progress but the risk of change is too high during this time. Typically development teams can work on prototypes or base platform features (such as utility libraries and such) without risking their development efforts from being rendered useless as major changes occur in core requirements.
2) Unknown Technical Challenges: In my many years of new product development, I have found the need to address unknown technical challenges as absolutely critical no matter which development methodology was being used. For a new product, the number of unknown technical challenges (ideas that have not been proven to work) is way bigger than for an established product. It I usually necessary to identify these challenges, understand their impact, plan contingencies and prove the ideas sooner than later. If one of these efforts ends up falling in the later phases of agile development, this could have devastating effects on the overall delivery plans for the product.
3) Incomplete System Architecture: This proves to be the biggest challenge that an agile development project faces. Typically any of the issues listed above could contribute to this but the eagerness to start working on a small piece of a project without knowing how it fits together with the bigger picture is absolutely dangerous. System architecture provides guidance and framework to make sure that the individual pieces of a new product are coming together the right way. Not having complete core architecture is like a bunch of people walking on their own paths with a “hope” that they are going to meet somewhere. In reality they may be going away from each other, never to come together. Agile methodology has provisions for bringing different parts of the project to sync-up together at the end of each phase. Having a framework to assess where the overall project is going and how it can come together is a key to ensure timely delivery of the entire product (especially when the product is being developed for the first time).
4) Lack of Framework for Continuous Integration: Agile methodology requires that there be a way to continuously integrate the features being developed. Not having a framework to integrate, test and verify that the features are working together as intended could seriously hurt the project and may make integration effort a whole lot more challenging than it needs to be.
Taking advantage of agile methodology to roll out a new product in shorter cycles is something every product team should strive for. However, there are several pre-conditions that are essential for a new product development and should not be ignored. Adding a common pre-start phase to agile development to ensure that the framework is in place usually helps kick off parallel development of small features and continuously release them to customers. A little care before the development is in full swing can help minimize risks to a project and also make sure that the whole product comes together well without any major surprises.