A while back on Slashdot there was an unforgettable post on who had the worst working environment starting with aerospace. Those of you who actually are rocket engineers will undoubtedly cringe, but in my simplistic understanding, cleaner aircraft have less frictional drag and better fuel efficiency, so the full time job of cleaning airplanes exists. Paying to eliminate drag as a way to save money seems to be worth it in a lot of different areas. For example, there is enough value in eliminating slow code that it’s worth it to use tools like perfmon and the profiler.
Which prompts the question – what kind of drag can we eliminate from software testing?
I’ve been thinking about this idea on my own but it’s not a new one. Searching around I found that Christian Buckley and Darren Pulsipher had already written an introductory article on “The Drag Coefficient of Test-Cycle Reduction” that offers some good suggestions of places to look at to reduce testing drag. I’m sure there are others who have addressed this topic that I haven’t discovered yet.
Call me old fashioned but as a full time tester, a big part of really nailing quality on the software projects I’ve been involved with seems to getting high throughput and large numbers of bugs looked at, evaluated, and fixed. When you’re in the zone with those big numbers you really start feeling the nontrivial frictional drag associated with each bug. The drag of people getting emotionally involved on a particular issue, failing to repro the bug, getting swamped in configuration or install issues, taking too long to log bugs, swamped in email threads gone haywire, developers who can’t get the bug to repro, burnout, requisitioning a machine, etc. I think a lot of times other people have predictable reactions to the information you provide to them as a tester, so a lot of times you can set up the process in the first place so that they will not react in a way that gets you out of the zone as a tester. Here’s a brainstorm of some ideas that come to mind for me, I’m certainly open to any other ideas people might have on this:
- Involving developers in test design up front, early, often so that you don’t have periods of large test debt to suddenly make up for, or big contortions to go to make it work.
- Time management (taking the time to make time and organize the schedule, we all know we should but for some reason we so often don’t create a system for doing it)
- Decluttering email (again, something that plagues so many of us because we don’t just take the time to create a system for managing it, not because we don’t know how)
- Mastering the art of logging bug reports that you won’t have to see again until they’ve been fixed (clear information, good repro steps, etc). If you have to keep helping multiple people through your bug it takes time away from other things.
- Queueing up similar tasks (like closing/verifying bug fixes in a particular build) that would otherwise require a lot of context switching and setup/teardown overhead.
- Replacing or fixing slow tools (like a bug logging tool that takes 1 minute to log each bug when you have just finished an exploratory test pass with a big list of bugs to add into the system, or like a test suite that has a huge timeout value for each test, unecessary setup/teardown for each test, UI tests that sleep() instead of getting event notification, a procedure that makes the test suite take ten times longer for no significant value gain, etc. )
- System for quickly getting your test machine to start state (need some fast way to reimage, bounce, uninstall, etc. the system so that you are back at a clean start state. Big problem if it takes you a day to set up the machine and a day to set up the tests. Virtualization, re-imaging, having a planned schedule, etc. seems to be things people are doing to help reduce drag this way)
- Having a reliable build system that gives you a testable versioned build when you need it rather than having to ask for a build or figure out how to get one, or build your own unversioned build that could be different from what others are testing.
- Having test sources next to product source so less time context switching.
- Some system for getting your brain to pick up where you left off the day before (too many details in our heads and not written down – meaning extra time to pick up where we left off and get going with the same amount of speed)
- Understanding pairwise testing and other ways to get smarter coverage for less work – I think this is more what the Buckley and Pulsipher paper was targeting.
- Remembering the agile manifesto as a motivation for having slick running processes and tools that you can set on cruise control allowing you to focus on individuals and interactions. I think this is totally compatible with the idea of eliminating drag because you are a lot more agile if your toolset isn’t constantly breaking on you.
- Triage meetings that bring everyone together to go through all high pri bugs in for example one hour vs. having everyone do it by email – bringing in the higher bandwidth of face-to-face and voice when it makes sense instead of email.
- Having some pride of ownership and fixing buggy test automation that you would otherwise have to continually investigate, over and over again, only to confirm that it was a test bug and not a product failure.
- Investing time to do your homework before starting discussions and email threads that are doomed to fail from the moment you hit send. Using modes of communication like privately talking to someone on the phone or in office first before involving the whole team. Mail that sparks a lot of discussion, generates more heat than light, pretty much predictably swings back and hits you in the face every time, and subsequently does nothing to help you get the product better tested or the bug closed.