Regression Testing
Regression testing defination and scope has always fascinated me even after 8 years in Testing World. What really amazes me is it hasnt evolved a lot since last time someone defined defination of regression testing some 20 years back. Software testing is one field where evolution has been really slow and stuck at books of my parent's generation.
So i started to think back and re-looked at defination of Regression Testing, top of world's top search engines bring up this page as first result (Bing and Google).
https://en.wikipedia.org/wiki/Regression_testing
I looked at history of page and it was last edited on 29-March-2012.
Lets look at defination...
Regression testing is any type of software testing that seeks to uncover new software bugs, or regressions , in existing functional and non-functional areas of a system after changes, such as enhancements, patches or configuration changes, have been made to them.
The intent of regression testing is to ensure that a change, such as a bugfix, did not introduce new faults.[1] One of the main reasons for regression testing is to determine whether a change in one part of the software affects other parts of the software.[2]
Common methods of regression testing include rerunning previously run tests and checking whether program behavior has changed and whether previously fixed faults have re-emerged. Regression testing can be used to test a system efficiently by systematically selecting the appropriate minimum set of tests needed to adequately cover a particular change.
I for some reason dont agree to this defination for reasons explained further, this sounds like assuming many things which have evolved in test space.
- Testing technique is mostly Blackbox
-Tester does not knowledge of internal system in terms of they dont look into code to know dependency of objects or dont do any impact analysis while writting Test Plan
- They wait till Regression test cycle at end of Test Phase to identify if system have any new bugs or regression bugs
I feel these assumptions are not so true nowourdays, with Test organizations moving towards more of SDET profile and more focus on White Box techniques like code reviews by Test team, Code Coverage techniques, Peer reviews, Inspections at all stages; testers are better equipped to analyze and make decision which areas to regress in the application rather than regressing the entire application. Let me try to explain what i mean with visual diagram.
Lets say your application consists of below modules from A to F, you had regression suite with say 300 test cases which have coverage for all areas, and it costs 10 days to run that regression suite.
Now in next release C has been modified to C++, which impacts A and F, due to indirect impact or data flow lets say it also impacts B not less than A and F. So below diagram shows areas which are impact due to this change in yellow or light yellow.
Now if we go by definations which had assumptions i listed earlier, i would still need same effort and resource for regression testing.
Here comes problem, i dont agree to that concept. I feel now with tools, techniques, more SDET skills, i know that only A, B and F are affected due to change in this release and it has no what so ever impact on E and D. So I should be testing only A, B and F in regression testing, which is going to save time for me and make my project manager more happy :-)
Ofcourse my approach assumes that you have the skills and knowledge to identify A, B, F as impacted areas with confidence in first place but thats where we are heading isnt it.. Testers are getting more skilled day by day with lots of focus on technical skills and domain knowledge.
We should get better at doing regression testing with evolution of testing space, get smarter and bring in more productivity every time we do something, do your regression cycles faster and target right areas which would yield potential bugs rather than executing same regression suite which was never re-visited and keep on taking standard days forever!!!
-Naren
Views or Opinions expressed in this blog represnt only author and not any company or organization or person.