the future of software testing (part 3)

So we are now at my third prediction which deals with information and how testers will use information to improve their testing in the future.

Prediction 1: Testsourcing

Prediction 2: Virtualization

Prediction 3: Information

What information do you use to help you test your software? Specs? User manuals? Prior (or competing) versions? Source code? Protocol analyzers? Process monitors? Does the information help? Is it straightforward to use?

Information is at the core of everything we do as software testers. The better our information about what the software is supposed to be doing and how it is doing it, the better our testing can actually be. I find it unacceptable that testers get so little information and none of it is specifically designed to make it easier to do our jobs. I am happy to say that this is changing … rapidly … and that in the near term we will certainly be gifted with the right information at the right time.

I take my inspiration for testing information from video games. In video games, we have the surfacing and use of information darn near perfected. The more information about the game, the players, the opposition, the environment, the better you play and the higher score you achieve. In video games this information is displayed in something called a HUD or heads up display. All a players’ abilities, weapons, health info are displayed and clickable for immediate use. Likewise, your location in the world is displayed in a small minimap and information about opponents is readily available (my son used to play Pokémon in which he had access to a Pokédex which kept information about all the various species of Pokémon he might encounter in the game … I’d like a Bug-é-dex that did the same for bugs I might encounter). The idea is simple: the more information you have and can use, the better your chances of success in the game.

I'd very much like to replicate this for software testers: give them more information to increase their success. But most of the testing world is mired in black box testing without such a rich information infrastructure. Where’s is our minimap that tells us which screen we are testing and how that screen is connected with the rest of the system? Why can’t I hover over a GUI control and see source code or even a list of properties the controls implements (and that I can test)? If I am testing an API, why can’t I see the list of parameter combinations that I and all my fellow testers have already tried? I need all of this information quickly and in a concise and readily consumable format that assists my testing rather than having to shuffle through some SharePoint site or database full of disconnected project artifacts. All this does is distract me. I want a heads up display!

My colleague at Microsoft Joe Allan Muharsky calls the collection of information that I want so badly a THUD – the Tester’s Heads Up Display – putting the information a tester needs to find bugs and verify functionality in a readily-consumable format for software testers. Think of a THUD as a skin that wraps around the application under test and surfaces information and tools that are useful in context of the application. Few THUDs are in use today or even contain the right information. In the future, no tester would think of testing without one, just like no gamer could imagine traversing an unpredictable and dangerous world without their HUD.

If this sounds a little like cheating then so be it. Gamers who add cheats to their HUD have an even bigger advantage over gamers who don’t. And as in-house testers who have access to the source, the protocols, the back-end, front-end and middleware we can indeed “cheat.” We can leverage these cheats to gain a massive bug-finding advantage over ordinary black box testers and users. This is exactly the situation we want: to be in a position to find our own bugs faster and more efficiently than anyone else. This is cheating I approve of wholeheartedly but today we’re not taking advantage of the information required for the cheats.

In the future, we will. That future will be fundamentally different than the information-starved present in which we are current working.