SysTest part III.: Introduction to test suites

In the first part I mentioned the following code:

  static void RunMySimpleTests(Args _args)
 {
    SysTestSuite suite = new SysTestSuite(classstr(MySimpleTests));
    SysTestResult result = new SysTestResult();
    ;
    result.addListener(new SysTestListenerPrint());
    suite.run(result);
    print result.getSummary();
 
    pause;
 } 

Simple suites

We didn't discuss this code in detail and we will not cover all the aspects of this sample even today. We will concentrate on the first line. Our test class usually contains multiple tests and to run them all it is easier for the framework to wrap them together (don't get me wrong, they are still independent UnitTests) and execute them at once. To do that SysTest contains SysTestSuite class.

What you see in the previous sample is creation of SysTestSuite class from MySimpleTests class. SysTestSuite constructor will take all test methods (see part 1 to understand what methods are considered unit tests) and add them to the newly created suite. To run all the tests now it is enough to just call run on the suite object.

Let's take a look at architecture diagram (this is extremely simplified diagram to show just the important parts):

You might recognize the Composite pattern there. Therefore it is possible not to add just tests into a suite but you can also add other suites. This way you can create pretty complex tree structure of suites (we might discuss later why that is a good idea).

We already know how to create suite from a test class but how do we add an individual test (or suite) to the suite? To answer this question we first have to understand how to represent (instantiate) an individual test.

  SysTest::newTestable(new MySimpleTests(), methodstr(MySimpleTests, testInfo))

I know what you think. This must be possible to do it simpler than this. Well, you're right but we don't have to worry about this too much now. Once we have the test we should be able to run just this individual test.

The following job runs just that one test:

     SysTestResult result = new SysTestResult();
    SysTest test = SysTest::newTestable(new MySimpleTests(), methodstr(MySimpleTests, testInfo));
    ;
    result.addListener(new SysTestListenerPrint());
    test.run(result);
    print result.getSummary();

Now let's get back to the original question. How do I add this individual test to my suite? Well, if it is just that one test then you can pass it to the suite constructor.

     SysTest test = SysTest::newTestable(new MySimpleTests(), methodstr(MySimpleTests, testInfo));
    SysTestSuite suite = new SysTestSuite(test);
    SysTestResult result = new SysTestResult();
    ;
    result.addListener(new SysTestListenerPrint());
    suite.run(result);
    print result.getSummary();

    pause;

What if now I want to create a suite containing two tests? First test will be testInfo from our previous MySimpleTests class and the other one will be testThrowErrorMessage from our previous MyExceptionTest class. We can't use suite constructor here but instead we will add individual tests using add method.

     SysTestSuite suite = new SysTestSuite();
    SysTestResult result = new SysTestResult();
    ;
    suite.add( SysTest::newTestable(new MySimpleTests(), methodstr(MySimpleTests, testInfo)) );
    suite.add( SysTest::newTestable(new myExceptionTest(), methodstr(myExceptionTest, testThrowErrorMessage)) );

    result.addListener(new SysTestListenerPrint());
    suite.run(result);
    print result.getSummary();

Run the test to see our tests running (use print listener to see execution details and if you use job then don't forget to add pause at the end).

Suites of suites

Now let's create a suite containing other suites. Let's say that now we want to add a suite of all tests from MySimpleTests, a suite of individual tests testThrowErrorException and testThrowException2, and finally an individual test testThrowErrorMessage. The following code should be familiar to you now.

  static void RunSuiteOfSuites(Args _args)
 {
    SysTestSuite simpleTestsSuite = new SysTestSuite(classstr(MySimpleTests)); // <-- MySimpleTests suite
    SysTestSuite exceptionTestsSuite = new SysTestSuite();                     // <-- MyExceptionTest suite
    SysTestSuite suite = new SysTestSuite();                                   // <-- root suite
    SysTestResult result = new SysTestResult();
    ;
    // Populate exceptionTestsSuite:
    exceptionTestsSuite.add( SysTest::newTestable(new myExceptionTest(), 
                             methodstr(myExceptionTest, testThrowErrorException)) );
    exceptionTestsSuite.add( SysTest::newTestable(new myExceptionTest(), 
                             methodstr(myExceptionTest, testThrowErrorException2)) );
 
    // Populate the main suite
    suite.add(simpleTestsSuite);
    suite.add(exceptionTestsSuite);
    suite.add( SysTest::newTestable(new myExceptionTest(), 
               methodstr(myExceptionTest, testThrowErrorMessage)) );
    
    // Run the suite
    result.addListener(new SysTestListenerPrint());
    suite.run(result);
    print result.getSummary();
 
    pause;
 }

Run the job and see that it really runs all your tests and that the tests are really executing from the right suite.

The previous code demonstrated how to create suite dynamically at runtime. Is it possible to do the same at design time? Let's create a suite class from the previous sample.

All we have to do is to create a class derived from SysTestSuite and override the constructor.

  public class MyFirstSuite extends SysTestSuite
 {
    public void new()
    {
        SysTestSuite simpleTestsSuite = new SysTestSuite(classstr(MySimpleTests)); // <-- MySimpleTests suite
        SysTestSuite exceptionTestsSuite = new SysTestSuite();                     // <-- MyExceptionTest suite
        ;
        super();
    
        // Populate exceptionTestsSuite:
        exceptionTestsSuite.add( SysTest::newTestable(new myExceptionTest(), 
                                 methodstr(myExceptionTest, testThrowErrorException)) );
        exceptionTestsSuite.add( SysTest::newTestable(new myExceptionTest(), 
                                 methodstr(myExceptionTest, testThrowErrorException2)) );
    
        // Populate the main suite
        this.add(simpleTestsSuite);
        this.add(exceptionTestsSuite);
        this.add( SysTest::newTestable(new myExceptionTest(), 
                  methodstr(myExceptionTest, testThrowErrorMessage)) );
    }

    public static void main(Args _params)
    {
        MySuite suite = new MySuite();
        SysTestResult result = new SysTestResult();
        ;
        result.addListener(new SysTestListenerPrint());
        suite.run(result);
        print result.getSummary();
    }
 }

When I load this test in my GUI runner I see the right structure.

The GUI runner used for the previous screenshot is not part of SysTest -- if there is interest we can discuss this runner in a later post. It allows you to run just a certain nodes and review execution messages per suite/test.

You can download these samples: Part03: Suites.xpo

It was a long post describing just how to create suites. In the next post I will describe why do we have suites and what are the different types of suites you can use.