Execution And Verification Are Tightly Coupled

Test cases are composed of three primary sections:

  • Cache initial state. The primary task for any test case is verifying that an action had the intended effect; if the test case is to know what changed it clearly must save off the starting state.
  • Execution. The test case must do something to have something to verify.
  • Verification. The test case uses the cached starting state and its knowledge about what it did to verify that the current state matches the test case’s expectations. This is perhaps the most important part of the test case, since it doesn’t matter how many hoops a test case sends the application through if the results are not accurately and thoroughly verified.

Using this strategy, knowledge of what test actions are to be executed become paramount to effective verification, so execution and verification each become tightly tied to the other. Indeed, a typical test case has this verification hard-coded within itself. This gives the test case everything it needs to execute; this also binds everything needed to understand or review the test case into the test case itself. However, the tight correlation between execution of the test case and verification of the test case makes reuse of either the execution steps or the verification nearly impossible. Additionally, any changes in the actions the test case executes generally require modifying the verification as well.

We have developed a technique that enables test case verification to be driven by actions being executed yet still be decoupled from the details of test case execution.