Software Testing should be challenge, creative and enjoyable

  • Article

​I believe the task of software testing should be creative and enjoyable.  In term of creativity of software testing,  we should never stop at the current testing methodology what we have today, we should always try to find a better way for testing, bringing ideas from other company, other fields.  In here, I list a couple of areas of software testing:

  • Using Game for testing.
  • CrowdSource Testing.
  • Use Artificial intelligence for software testing/verification.  If you have chance to look at Bing team's XRay testing and Heuristics based testing, the idea and result is very good.
  • What about hardware testing? What we can learned from hardware testing.
  • What about testing airplane? How Boeing testing their airplane?

Once we encourage people to be creative, you will find out that testing is enjoyable and you will like testing. Of course, Software testing is always challenge which is one of the reason we can keep talent in testing area.

In the end, I like to share one conversion about scenario based testing and game testing to you: 

From: XXX
Sent: 04 March, 2012 10:54 PM
Subject: Anyone can tell me, is there any relation between scenario testing and game testing ?

 

I m a Embedded windows Components tester , and next step will be attending into Scenario testing work . I am not so familiar with this Testing skill which called Scenario testing . Before came MS, I was a game tester in Gameloft , and after read a lot of docs about scenario testing I found that ,maybe there are some relationships between game testing and Scenario testing .

In game testing , there is no case database, because game testing is a large randomly work . Of course , the basic functional testing is necessary such as use W A D S key to move MC(Main Character),space to jump and so on . The important thing is , ALL Test case are stored in tests mind, that I mean , game testing doesn’t need , or cannot write test case to test . because all bugs are triggered by scenarios.

Like ,A Game on Touchable devices .
User used Virtual joystick to move MC forward to the building named ” Death Sword “, then sit on the best left chair , Meanwhile ,The user tapped setting button and turn off the music .

Even if this scenario is for testing . The Music could be turned off anywhere and anytime as user’s mind. However , in that situation , maybe there is no code covered which leaded a crash or high priority bug.

These are about game testing , back to Windows Scenario testing , I have some questions ,

1.Do we need to write a lot of cases like feature testing , maybe some “Scenario test cases.“ (I can’t even confirm whether this term is existent) .if you had wrote some , please tell me whether this term is existent , and would you please give me some examples.

2.I wrote a game testing process above , in my opinion , the scenario is something like a motivated bug report . you can even use it directly in your bug description , which explain to the dev “ This is a costumer really concerned issue , you’d better to check it out “ , so when we write some scenario , we need use a story maybe a little like the Story above to replace some Step by step cases, right?

3. Anyone who had been a game tester , do you agree with my opinion – there is something relation between scenario testing and game testing ? anyway , Please tell me something about scenario in your mind , if you familiar with it .

I am so expecting you guys replies , u know, when I relate the scenario to the game testing , that feel such interest in works in the future .

 

From: Qingsong Yao
Sent: Tuesday, March 06, 2012 1:06 AM
Subject: RE: Anyone can tell me, is there any relation between scenario testing and game testing ?

 

Hello, Ting
  What you said is really valuable and correct.  That is true that sometime it is pretty hard to capture user scenario or when you write\automated your scenario tests, it lost the value.  For example, this blog at
https://www.developsense.com/blog/2010/05/why-we-do-scenario-testing/
is a very good article. And the author concluded at
“One of the lessons to be learned … is that the sheer number of tests performed is of little significance in itself. Too often, the series of tests simply proves how good the computer is at doing the same things with different numbers. As in many instances, we are probably misled here by our experiences with people, whose inherent reliability on repetitive work is at best variable. With a computer program, however, the greater problem is to prove adaptability, something which is not trivial in human functions either. Consequently we must be sure that each test does some work not done by previous tests. To do this, we must struggle to develop a suspicious nature as well as a lively imagination.“

About your gaming testing idea, that is the right way to think about how we do testing.   There are quite a couple of similar ideas which you might explore:
• The Future of Testing Talk by James A. Whittaker  which have one section about how Xbox team testing games.
• Exploratory Software Testing is always a good idea. You can look at Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design by James A. Whittaker which has lot of cool ideas about testing.
• You can also look at https://specflow.org for the Acceptance Test-Driven Development (ATDD) and follow the links to a couple of articles/books about why we do Acceptance testing.
• Recently we had a talk about “The future of testing is play” which is about using Game for testing.

 

Thanks
Qingsong