Why do IT Projects Fail?

I read an interesting article in this morning’s Ottawa Citizen. It says the auditor general has stated the government is mismanaging a number of IT projects. The auditor general’s office found a history of cost overruns and delays and of not delivering on the project’s planned purpose. There are a myriad of reasons that can cause a project to run over budget or miss deadlines. I’d like to hone in one of that one specific point from the article “Not delivering on the project’s planned purpose.”

This one hits close to home for me because for the past few years I was training not just developers but also IT business analysts. A business analyst gathers the requirements for an IT system. A systems analyst on the other hand determines the specifications for an IT system. What’s the difference?

Requirements describe WHAT the user needs from the system. For example, the user needs to be able to know who reported a bug and needs to be able to track the progress of the bug they reported.

Specifications describe HOW the system will meet that need. For example, there will be a SQLServer database and a User table with a ReportedBy field of type VARCHAR, there will be a mobile application which will allow users to view the status of their bugs written using Silverlight and a web service.

Some projects have dedicated business analysts who will gather the requirements and hand them off to the technical team. On other projects members of the technical team are asked to gather the requirements, Sometimes, requirements gathering is skipped and we just start building because “we know what the user needs” You can be successful gathering requirements with business analysts or technical team members but you should never skip requirements gathering because you think you know what the user needs. Even in the agile methodologies the user is the one who drives the content in the sprints.

With our technical skills we are very good at building solutions to meet a specific need or requirement. I have been blown away over and over again with the ability of programmers and system architects to come up with creative ways to meet users needs even when faced with technical limitations. But, if the users needs aren’t well understood in the first place you are headed for trouble.

Indulge me for a moment as I relate a short story from my own work life. I was managing a few programmers and was asked to build a small application for another team to use internally. I sat down with the team lead who wanted the system for about an hour to get a feel for what they were after. Then I walked away, sat down with my team and we designed a solution to meet their needs. That first meeting was the only meeting I had with the other team. We were on a project that was already behind schedule and over budget so there was a lot of pressure to get this application out the door. Since it was an internal application there was no requirement from management for lengthy documentation or sign off. With a lot of long hours and creative programming we turned around the application in two weeks! We were patting ourselves on the back for a job well done, especially with the time constraints! We walked into a meeting with our customers (the other team) to show them this wonderful new application we had built for them. Within 5 minutes it was clear that although our application worked fine, it did not meet the other team’s needs! They got angry because the system didn’t meet their needs. We got angry because we had just killed ourselves for two weeks getting this completely functional solution built and out the door. The result was increased tension and anger on a project that was already under stress and a solution that didn’t actually help the other team’s productivity. Why did it happen? Because we didn’t fully understand the requirements!

I can’t teach you how to correctly gather requirements in one blog post, I can tell you gathering requirements is a critical success factor for any project and requirements gathering is the theme for today’s My 5

My 5 tips for successful requirements gathering

1. Your requirements will be wrong – No matter how much time you spend on requirements, no matter how many users you talk to, and how many meetings you hold, you cannot get the requirements 100% correct. That doesn’t mean you should just skip requirements gathering because the requirements will be wrong, it just means you need to accept the fact you can’t get it 100% right.
2. When gathering requirements your goal is to find as many mistakes in the requirements as you can with the time you have – Since you can’t get requirements 100% correct, you want to get as close as you can to 100% with the time you have. How do you find mistakes in the requirements? See Number 3.
3. Engage the users early and often - Users will forget to tell you something in that first meeting, or a different user will be aware of a requirement that another user will miss. Engaging different users, and following up with emails, phone calls or additional meetings increases your chances of finding mistakes or gaps in the requirements
4. Get requirements signed off before you start testing – It is not unusual for a project to start development when the requirements are still in flux, but how can you test to see if requirements are met if you haven’t agreed on the requirements yet?
5. Document in one place only – Since the requirements are wrong, you know you will have to update them as you find the mistakes and gaps. Get creative in how you document your requirements. If you find yourself doing a cut and paste from one requirements document to another, ask yourself how can I document this in one place so that *when* it changes, I only have to update the requirement in a single place.

For more information about requirements gathering, I suggest you check out The International Institute of Business Analysis (IIBA). They organize conferences on requirements gathering, and they even have a business analysis certification based on the information in the Business Analysis Book of Knowledge (BABOK) which is chock full of best practices for gathering requirements. Some of the tips in todays My 5 come from material prepared by Noble Inc.