test case reuse (in the future)

I’ve given my ‘future of testing’ talk four times (!) this week and by far the part that generates the most questions is when I prophesize about test case reuse. Given that I answered it differently all four times (sigh), I want to use this space to clarify my own thinking and to add some specifics.

Here’s the scenario: one tester writes a set of test cases and automates them so that she can run them over and over again. They are good test cases so you decide to run them as well. However, when you do run them, you find they won’t work on your machine. Your tester friend used automation APIs that you don’t have installed on your computer and scripting libraries that you don’t have either. The problem with porting test cases across machine boundaries is that they are too specific to their environment.

In the future we will solve this problem with a concept I call environment-carrying tests (nod to Brent Jensen). Test cases of the future will be written in such a way that they will encapsulate their environment needs within the test case using virtualization. Test cases will be written within virtual capsules that embed all the necessary environmental dependencies so that the test case can run on whatever machine you need it to run on.

The scope of technological advances we need for this to happen are fairly modest. However, the Achilles heel of reuse has never been technological so much as economic. The real work required to reuse software artifacts has always been on the consumer of the reused artifact and not on its producer. What we need is an incentive for testers to write reusable test cases. So, what if we create a “Testipedia” that stored test cases and paid the contributing tester, or their organization, for contributions? What is a test case worth? A dollar? Ten dollars? More? Clearly they have value and a database full of them would have enough value that a business could be created to host the database and resell or lease test cases on an as-needed basis. The more worthy a test case, the higher its value and testers would be incentivized to contribute.

Reusable test cases will have enough intrinsic value that a market for test case converters would likely emerge so that entire libraries of tests could be provided as a service or licensed as a product.

But this is only part of the solution. Having test cases that can be run in any environment is helpful, but we still need test cases that apply to the application we want to test. As it turns out, I have an opinion on this and I’ll blog about it next.