Creativity in Testing

  • Article

One of the wonderful parts of testing software is that beyond the pure technical aspects of the discipline lie many opportunities for the tester to exercise their creativity in potentially game-changing ways.  Although we work in a field where some of the quality assurance tasks are handed down to us from previous owners (legacy test suites, existing test cases, etc.), it is important to note that it is possible to break from the norm and provide real value by using approaches that have never been used before in the areas which we are testing.  Sometimes we can see greater gains not from working harder, but by simply thinking differently.  Some of this takes courage, as it is hard to differ from some of the time honored approaches.  However, as my experience grows in the field, time and time again I find that the testers who distinguish themselves are not afraid to try new techniques and approaches, and are always looking for creative ways to break the product and new ways to improve.   Sometimes these new techniques end in failure, and that is ok.  Sometimes the new techniques spur on measurable improvements in the ways that we look at and validate the quality of the software.

 

There are times when testers are viewed simply as “test execution machines”, but great value can come out of looking beyond what is scripted or specified and going “beyond the tests”.  What we do is more of an art than a science, and that is what keeps it interesting to me personally.  Sometimes it can feel great to hear the words from your teammates "How did you think of doing that?"  J

 

What are some ways that you are able to spark your testing creativity and/or find inspiration?  Here are a few ideas that we have used in teams that I have worked in.

  • Randomly pick two features in the same product and look for all intersections between them; test those intersections
  • Use the software without a mouse
  • Test on extremely low-end hardware or extremely low bandwidth network connection
  • Learn from all of your predecessors.  Look at old bug reports, you will find a wealth of untapped information and ideas which can help you with new approaches.  Also look at reports from unrelated areas…
  • Use the software with your "off" hand 
  • Take your work outside   J  Test outside, on your laptop
  • Gain inspiration from other disciplines.  Sometimes use of creativity in art, music, business, athletics, etc. can help influence creativity and provide a spark in how you go about testing.  Research your favorite musicians, athletes, carpenters, humanitarians, and see what inspires them, see what makes them tick.  As an example:  Jazz musicians will frequently engage in improvisational music in which the instrumentalists feed off of the spontaneous melodies of their band-mates.  This same technique can be applied to testing…get a small group of folks in a room together and have them “group-test” a product, supplying little structure/guidance to the testing.  Allow the testing to bounce from one area to another and see where it takes the group…sometimes the results can be very interesting… 
  • Test a feature side-by-side with a competitor's offering
  • Go for a quick jog, and brainstorm test cases along the way.   You will remember some of them  when you get back  J
  • Use the software for a day with a very large font setting
  • Write a test suite which generates data based off of customer interaction patterns, gathered by telemetry
  • Artificially constrain your time – role play the scenario where you only have a single hour to ensure the quality of your feature.  What would you test?  What would you try?  Where would you spend your time?
  • Have your children and/or parents try to break the product.  Watch them as they use it, and note any usability or functional concerns
  • Use the wisdom of others to help test your software.  Have a large population of engineers vote on whether particular features work correctly or not.  Use this data to target testing deep-dives
  • Share with others – sometimes just talking through some of the more interesting bugs you have encountered with a few folks informally can start important testing collaborations
  • Have someone else give you a list of things that are important to them in the software, and then use it as a starting place for your own exploration
  • Try model-based testing in an area which has never been tested that way before
  • Without looking at specifications, come up with a list of "Ten things that must work" for your feature/area; verify that these things indeed work
  • Test someone else's area while they test yours
  • Step through the code in the debugger while you are testing a scenario
  • Read the code flow itself while you are testing a scenario
  • Tape a golf ball to your keyboard (yes, this was done  J )
  • Test while listening to mariachi (or speed metal, classical, jazz, etc) music
  • Etc.

 

Comments/Questions are always welcome.   Please contribute/share your own examples of inspiration and/or creativity.  Thanks for reading!

 

-Liam Price, Microsoft