SysTest part VI.: Code coverage

Article
03/07/2007

In this post I want to describe code coverage module in SysTest framework. Let's have a simple calculator class (calculating area):

  public class SimpleCalc
 {
    public int square(int _a)
    {
        int square = 0;
        ;
        if (_a > 0)
        {
            square = _a * _a;
        }
        return square;
    }
    public int rectangle(int _a, int _b)
    {
        int rectangle = 0;
        ;
        if (_a > 0 && _b > 0)
        {
            rectangle = _a * _b;
        }
        return rectangle;
    }
 }

Now let's have the following TestClass:

  public class SimpleCalcTest extends SysTestCase
 {
    public void testRectangle()
    {
        SimpleCalc calculator = new SimpleCalc();
        ;
        this.assertEquals(0, calculator.rectangle(-1, -1));
        this.assertEquals(0, calculator.rectangle(0, -1));
        this.assertEquals(1, calculator.rectangle(1, 1));
        this.assertEquals(6, calculator.rectangle(2, 3));
    }
    public void testSquare()
    {
        SimpleCalc calculator = new SimpleCalc();
        ;
        this.assertEquals(0, calculator.square(-1));
        this.assertEquals(0, calculator.square(0));
        this.assertEquals(1, calculator.square(1));
        this.assertEquals(4, calculator.square(2));
    }
 }

To run this test we use the following job:

     SysTestSuite suite = new SysTestSuite(classstr(SimpleCalcTest));
    SysTestResult result = new SysTestResult();
    ;
    result.addListener(new SysTestListenerPrint());
    result.addListener(new SysTestListenerDB());
    suite.run(result);
    print result.getSummary();

Notice the line where we register SysTestListenerDB listener. This listener writes all the execution data into SysTest database. You can browse this database either directly (not recommended) or using our UI from Tools > Development tools > Unit test > Test jobs. Locate your test job (the highest Test job ID -- there is a bug in the form and the form is not sorted correctly), and click Tests button. You see all test classes and their test methods, their result and code coverage. Note that the code coverage is 0%.

Now let's make a simple tweak and add one line of code to this job:

     SysTestSuite suite = new SysTestSuite(classstr(SimpleCalcTest));
    SysTestResult result = new SysTestResult();
    ;
    result.addListener(new SysTestListenerPrint());
    result.addListener(new SysTestListenerDB());
    result.parmCodeCoverageEnabled(true);       <-- CODE COVERAGE ENABLED
    suite.run(result);
    print result.getSummary();

Run the job again and note that in SysTest tables we now see 100% code coverage. See that if you select a test class or a test method and click on Coverage button, you can see the lines that were executed. Now you might ask two questions:

Is there a way to turn code coverage on/off somewhere in UI?
How does the framework know that the test class is supposed to test exactly our SimpleCalc class?

Answer to the first question is easy using Tools > Development tools > Unit test > Parameters where you can check Record code coverage checkbox. Answer to the second question is more difficult. The framework of course doesn't know if you don't specify the target class name. On the other hand the framework might try to guess. The framework sees that your test class is named SimpleCalcTest. If you don't specify anything then the framework will try to see if there is a class named SimpleCalc (the test name without "Test" suffix). If there is a class with this name then the code coverage will be calculated against this class.

Let's now assume that we have the same class but our tests should be named ScienceCalcTest. We now need to tell the framework that even though the test is named ScienceCalcTest the code coverage should be calculated for class SimpleCalc. To do so we have to override two methods:

     public identifiername testsElementName()
    {
        return classstr(SimpleCalc);
    }
    public UtilElementType testsElementType()
    {
        return UtilElementType::Class;
    }

The first method specifies that the target is named SimpleCalc and the second method specifies that it is a class. You can also target a table if you want to. Now update the job and run it.

     SysTestSuite suite = new SysTestSuite(classstr(ScienceCalcTest));
    SysTestResult result = new SysTestResult();
    ;
    result.addListener(new SysTestListenerPrint());
    result.addListener(new SysTestListenerDB());
    result.parmCodeCoverageEnabled(true);
    suite.run(result);
    print result.getSummary();

If you go to the SysTest table you should see that reported code coverage is again 100%. That's all.

Before I finish this post I would like to note that code coverage works by turning on DynamicsAX profiler. This causes the execution to be much slower. To demonstrate that let's run the following sample job:

     SysTestRunner test = null;
    int startTime = 0;
    ;
    print 'CodeCoverage: OFF';
    startTime = WinAPI::getTickCount();
    test = SysTestRunner::createTestRunner(classstr(SimpleCalcTest));
    test.getResult().addListener(new SysTestListenerDB());
    test.getResult().parmCodeCoverageEnabled(false);
    print test.run().getSummary();
    print strfmt('Elapsed time: %1 ms', int2str(WinAPI::getTickCount() - startTime));
    print '';
    
    print 'CodeCoverage: ON';
    startTime = WinAPI::getTickCount();
    test = SysTestRunner::createTestRunner(classstr(SimpleCalcTest));
    test.getResult().addListener(new SysTestListenerDB());
    test.getResult().parmCodeCoverageEnabled(true);
    print test.run().getSummary();
    print strfmt('Elapsed time: %1 ms', int2str(WinAPI::getTickCount() - startTime));
    print '';

Notice that the code coverage slows down the execution considerably. Therefore if you do test driven development, refactoring or any other development where you execute your tests very frequently, turn the code coverage off.

You can download these samples: Part06: CodeCoverage.xpo

SysTest part VI.: Code coverage

Additional resources