Microsoft Test Runner series – part 4 – Fast Forwarding

Article
01/12/2010

Hi again,

In one of the previous blogs we had talked that to reduce the time spend on less important tasks, MTLM provides a functionality called Fast Forwarding. The goal of this feature was to allow testers to quickly go to the step in the test case where they want to focus their testing during this test execution. It may be while making sure that the bug is real Or it may be during regression testing etc.

History: So how did it came into existence: When we went on customer visits, I noticed that a tester who just found a bug. However, before deciding to file the bug, he actually performed ALL the steps again (couple of times) to make sure if it a real bug. When asked for more details, he mentioned that although first few steps were not important for this bug, he had to perform these to reach this point. Note that the first few steps had a lot of data enter & hence it was taking time. Another tester did the same thing while verifying a bug that was fixed in last night’s build.

Talking to various testers we found that there are many scenarios, where performing certain steps are necessary to reach the point in test case where they want to focus their energy. They wished if they had magic wand to reach the step they wanted to test more thoroughly. When asked, why can’t you automate these sections that are not important but are necessary, the answers we got were:

- It is not easy to automate the tests/portions require skill, tools & time

- The UI keeps changing very often in initial phase of development & ROI is not worth.

- In many cases, it is not possible to plan which areas of tests should be automated e.g. for regressing a bug, tester can’t plan ahead what steps should be automated.

So we started thinking what can be done to reduce their pain. We came up with the idea of using record & playback in such a way that it can help testers in such scenarios. We knew that R&P has its own limitations; however, using it in this scenario fits very well, where a human is sitting in front of the screen. Also internally we had a technology that was being used successfully & had functionality to overcome basic issues with R&P.

Fast forwarding: The idea - When tester is testing a test case manually, record UI actions in the background (no extra overhead for the tester). One of the primary goals was to keep the manual testing as is i.e. manual tester should not be asked to do something different for creating the action recording. However we don’t want to produce one big recording for the whole test case, so it was decided that we find a way to sync what tester is doing on AUT & what manual test step is being performed. The idea of marker segments came into existence. i.e. as soon as tester marks a step result (pass/fail), the tool would know that testing this step is complete & hence create a boundary for the recorded actions. So more steps are marked by the manual tester the granular the action recording boundaries become.

When test is completed, the action recording just created would be attached to the test case. When tester tests this test case again, the action recording is available for fast forwarding any steps. Now tester does not have to perform the same steps again that she does not want to (but have to). She can simply select the steps (or or the barker region) & choose to play.

The playback engine will perform the same actions that were performed while testing is manually the first time. When playback is done, manual tester can perform the steps manually. The best part is that after performing few critical manual steps, she can again choose the rest of the steps & play them for fast forwarding.

Isn’t it great to be able to fast forward whatever steps you want, whenever you want!!!.

A tester can also choose to select all steps & play (just watch the actions being performed on the AUT). I know what you are thinking, but remember, you are a manual tester; don’t overdo it (your job is to find bugs) :)

In the next blog we will talk about what happens if I want to test the same test case with different data set & how fast forwarding can help in that scenario.

Microsoft Test Runner series – part 4 – Fast Forwarding

Additional resources