There's No Room For Watermelons
I have noticed a problem lately that needs some focus. I heard a statement that sums it up nicely. "Don't be green on the outside and red on the inside." Now if this was a joke (What's green on the outside and red on the inside?") I can think of a few answers like a frog, an alien (of course, maybe they are blue on the inside, who knows?), or a watermelon. But this isn't a joke. It's a serious problem in engineering. It's the problem of reporting the status that you think managers want to see instead of reporting the actual status of a project.
At Microsoft, we report measurements to determine if we are ready to ship a product. We typically color-code these measurements or the overall status derived from them using red, yellow, or green. Red typically means there are concerns and issues with the project. Green means that everything is on track. And yellow just means there are some issues but it doesn't warrant a green or a red. This color-coding is easy to understand and makes reports more readable.
At times, we've even added trending arrows. For example, let’s say a measurement target should be above 98%. If the measurement last month was 98% and is now 99%, then there would be a green up arrow next to this month's number. Green represents it being above the target and up represents a positive improvement. If instead it was 97% this month, there would be a red downward arrow next to that number. This is a great way to see both trending and status. (See arrows in picture below.) But be careful not to report on too much data. If there are no clear actionable items from the report, especially when an item is marked red, then you should question whether that information is useful to report at all.
Teams using the watermelon approach for reporting can create problems. For example, let's say a test team shows all green for % of test cases that are automated (vs. manual). When digging a bit deeper it is discovered that they are reporting higher test automation numbers because they exclude a bunch of test cases from the suite they are automating (tests that would typically always need to be run manually, like accessibility tests). Although the green status will cause management to think highly of the team's ability to ship products and they won't drill down on details within those projects, the green status will then cause problems in the future. Also within a group of many different test teams, having one team report overly green status causes other teams who were being accurate and honest with their data to get unnecessarily scrutinized and unfairly compared to this all green team. Over time, this watermelon approach will cause problems for this project team. The quality of their releases will go down and their request for more resources and additional help will be questioned because they never socialized that need with the use of red status.
A team also needs to be accurate in their reporting for the sake of the metric. If the metric isn't the correct one to be using to measure status and is instead driving the wrong behavior, having teams report they are green on it only supports that metric more. Behind-the-scenes they will continue to struggle to meet it. Joining other teams in being accurate and reporting red when something is truly red helps management look more closely at each measurement and course-correct as needed.
Teams who give all green status are all very afraid that going red will hurt their performance reviews or that they will look like incompetent people by being red. I have never seen that be the case. On the contrary, it takes a team with a lot of confidence and integrity to own up to falling behind or needing help. And sometimes, it's just the complexity of the project or technology that causes the red status. Also, giving your status as red and then explaining your plan to get out of the red shows more leadership skills than hiding your problems behind green status. For example, my team always reports as accurate data as possible. When our automation numbers were low, I could explain this because we were using new technologies that had no easy way to be automated. My team also created an automation strategy plan showing 6-month, 12-month, and 18-month automation goals based on our best forecasting on getting test hooks into the code, ramping up more people to write automation, etc.
All levels of management want to help out when necessary so be upfront about whether your team is on track or not. Managers don't want to be blind-sided and they appreciate open communications about whether their team will be able to ship a project on time. There are many reasons why a project falls behind and most are not due to incompetence yet it seems like people using the watermelon approach are afraid of being labeled incompetent. That's a myth that I wish we could expose as false. Is your team red on the inside and green on the outside?