SysTest quick start


Overview


I am a big fan of TestDrivenDevelopment and I am trying to follow this approach for all my development. Unfortunately DynamicsAX didn't have any UnitTest framework and thus it was difficult to use TestDrivenDevelopment. It was clear that we needed something in DynamicsAX and that's why we now have SysTest.


SysTest


SysTest is a UnitTest framework. If you are familiar with any other UnitTest framework then you will feel comfortable with SysTest.


Main features/goals:



  • it is very fast
  • it offers rich set of "assert" methods
  • it supports tests throwing exceptions (expected exceptions)
  • it supports transactions (tests can be placed inside a database transaction that is aborted at the end)
  • it supports company accounts (tests can be placed inside a separate company account that is deleted at the end of the test)
  • it supports fixture setup and tear down that are called automatically by the framework
  • it supports suites of tests
  • it supports setup and tear down method for suite that is run before the first method and after the last test method
  • it supports suites of suites
  • it supports code coverage
  • it has a wide variety of listeners for DynamicsAX environment
  • toolbar for quick test suite execution

Examples


Let's demonstrate the framework on examples. Let's create a unit test to test int2str conversion method. First we have to create our test class:

public class MySimpleTest extends SysTestCase
{
}

Now if you use toolbar (Tools > Development Tools > Unit Test > Show toolbar), then enter name of your test MySimpleTest and click Run. You immediately see "0 run, 0 failed". It's that simple. Ok, it maybe simple because it doesn't do much. Let's add a test validating that the function can successfully convert positive and negative numbers. To do so just add new method named with test prefix that is public, returns void and doesn't take any parameter. In the method check all your asserts.

public class MySimpleTest extends SysTestCase
{
public void testConversion()
{
this.assertEquals('123', int2str(123));
this.assertEquals('-2', int2str(-2));
}
}

When you run the test in a toolbar runner for example you will immediately see "1 run, 0 failed". Great! Let's add more tests to our test class. Let's see how the framework handles tests that throw exceptions but those exceptions are expected. Because of that we don't want to fail the test method. There are two ways to do that. First one is that you will code the exception handling yourself:

    public void testExpectedException()
{
try
{
throw Exception::Error;
}
catch(Exception::Error)
{
return;
}
this.fail('An expected exception wasn\'t thrown!');
}

As you can see when we expect an exception and it is not thrown then it is considered a failure. SysTest offers a better way to do the same.

    public void testExpectedException()
{
this.parmExceptionExpected(true);
throw Exception::Error;
}

Exceptions are more complex in DynamicsAX. Usually we don't just throw an error exception but we throw an error message. How do we handle that in SysTest?

public void testExpectedError() { ; this.parmExceptionExpected(true, 'Oops -- error message'); throw error('Oops -- error message'); } }

When you run the test you will get "3 run, 0 failed". Let's now see what happens when a failure occurs. We will begin by adding a failing test.

    public void testFailure()
{
this.assertEquals('0', int2str(123));
}

This is clearly a failing test. When we run the test class now we get "3 run, 1 failed". Toolbar displays all failures in Infolog window. For our test class we would see the following window:







On the other hand we don't have to rely on toolbar. Let's say that we want to see all messages (not just failures) and let's say that we want to write them into DynamicsAX Print window. First let's write our own runner. For a runner we have to specify the whole suite we want to run. That's easy and all you have to do is to create instance of SysTestSuite class and pass your test class name to SysTestSuite constructor. To run the suite call run method. Here is one little problem. Suite itself is not interested in results at all. The problem is that we are and if we want to know the result at the end we have to create special SysTestResult object and pass it to the suite when it runs our test.

    public static void main(Args _params)
{
SysTestSuite suite = new SysTestSuite(classstr(MySimpleTest));
SysTestResult result = new SysTestResult();
;
suite.run(result);
print result.getSummary();
}

The code above creates the suite for our test class and executes all the tests. At the end it prints the summary report to the Print window. If we want to print all messages (failures, information, but also when a test is started or completed) then we have to register the corresponding listener.

    public static void main(Args _params)
{
SysTestSuite suite = new SysTestSuite(classstr(MySimpleTest));
SysTestResult result = new SysTestResult();
;
result.addListener(new SysTestListenerPrint()); // <- LISTENER REGISTERED
suite.run(result);
print result.getSummary();
}

In total our whole class looks like this:

public class MySimpleTest extends SysTestCase
{
public void testConversion()
{
;
this.assertEquals('123', int2str(123));
this.assertEquals('-2', int2str(-2));
}
public void testExpectedException()
{
;
this.parmExceptionExpected(true);
throw Exception::Error;
}
public void testExpectedError()
{
;
this.parmExceptionExpected(true, 'Oops -- error message');
throw error('Oops -- error message');
}
public void testFailure()
{
;
this.assertEquals('0', int2str(123));
}

public static void main(Args _params)
{
SysTestSuite suite = new SysTestSuite(classstr(MySimpleTest));
SysTestResult result = new SysTestResult();
;
result.addListener(new SysTestListenerPrint()); // <- LISTENER REGISTERED
suite.run(result);
print result.getSummary();
}
}

That's all. If you want more information then stay tuned. In the next post I will try to describe individual features of SysTest in more details.

Comments (22)
  1. pyll says:

    Good job on documenting how this works in Ax.  When I was first looking at the SysTest* code, it looked like it wasn’t functional.  However, after delving further it looked better.

    I’m glad that it works.

    As for the code snippets you used above, they seemed to work, except for the main method.  I’m not sure if it is because I just threw the code into a job and ran it, but running it this way seems to produce no failures, eventhough the toolbar shows that there are errors…

    Any ideas?

    Thanks!

  2. Björn says:

    It’s great to see support for unit testing in the standard application.  It sure beats my feeble attempt of porting JUnit to Axapta 3.0.

    What still bothers me is that the standard application is not well suited for unit testing.  This is a problem when trying to customize the standard code in a TDD fashion.  You need a lot of setup code and data to test even minor modifications.

    It’s great for completely new modules though.

  3. mugz says:

    what portion of existing ax code is covered by unit tests?

    does MS internally do TDD and in what scale?

    How many tests are in the Ax4 codebase?

  4. Tests in Ax4 codebase says:

    I’ve been playing with one of the pre-releases of 4.0 and don’t think there are any unit tests in the standard codebase.  Not even for the test framework itself.

  5. dpokluda says:

    Sorry for such a late reply. I’ve been out of the town. Here are the answers to your questions:

    Re Pyll: Just add the main method to your test class. Then you can open the class — this would execute the main method.

    Re Bjorn: You are right. Standard (legacy) applications were not designed with testability in mind. Therefore it might be hard to test those.

    Re Mugz: Core of SysTest was done using TDD. There are other teams that are using TDD (like test automation framework team for example).

    Re Tests in Ax4 codebase: We have already thousands of unit tests covering mostly the new functionality. For 4.1 development we have even higher bar and there must be certain code coverage before you can check-in the code (otherwise the check-in would be refused). These tests are not shipped with the product and so you will not find them.

    I’m planning to post another post either later this week or early next week. 🙂

  6. Björn says:

    Too bad no tests are included.  It would be good to have them both for testing if a modification breaks existing functionality, as well as having an example how to use existing code.

    Can you tell us why they aren’t shipped with the final release?

  7. dpokluda says:

    Re Bjorn: I totally understand what you’re saying and I agree that it might be a help for all the partners if they would have some of our tests. On the other hand I also understand others saying that the value of shipping the tests wouldn’t be so big since with all the modifications (made by partners) most of the tests would either not compile or fail anyway. Partners would either have to invest heavily to update all the tests (and repeat that investment with every release) or to investigate every failure to find out whether it is a valid failure or not.

    On top of that you have to understand that there is different bar for code quality for tests. Therefore we don’t really want to publish/ship those classes with the product. I remember that there were some thoughts about shipping our test automation framework with one of our test suites but since test automation framework doesn’t meet our shipping code quality bar we never did that.

    I think it’s great we’re shipping the framework now and people can define and write their own set of tests. We might also try to define a basic set of tests and ship those later but that’s not something I can or will decide. 🙂

  8. Björn says:

    Thanks for the clarification, David.  Those are of course good arguments.  I’m just one of those guys who wouldn’t mind dealing with broken tests and "low quality" code 🙂  IMO it would increase overall quality in the long run.

    I agree that it’s great to finally have a testing framework. And it’s looking very promising. No complaints there. Can’t wait to use it on a real project.

  9. Dom says:

    Thanks alot for writing this article~~!

    By the way, I agree with Bjorn regarding availability of some samples.

  10. dpokluda says:

    I understand what you’re saying and I also think it would be useful for you to have some of our tests. Try to be heard and maybe the policy will change. 🙂

  11. ... says:

    Great site! Good luck to it’s owner!

  12. ... says:

    pagine piuttosto informative, piacevoli =)

  13. ... says:

    luogo grande:) nessun osservazioni!

  14. ... says:

    Great site! Good luck to it’s owner!

  15. Ashok Kumar says:

    I am a newbie to axapta, Please give me the clarity on SysTest Framework.

  16. Anh Mai says:

    Do you have any idea about tool to test for kernel class (use C++ language).

    Because i think this tool is used for test on X++ layer.

    Thanks!

  17. John L Bevan says:

    For those following this thread, request for in house tests made to Microsoft Connect: connect.microsoft.com/…/include-automated-tests-with-dynamics.  Please vote if you're in favour of this idea; thank-you.

Comments are closed.

Skip to main content