Automated testing guidance for AX 7
Couple of links that will be helpful while thinking about testing in general
- https://blogs.msdn.com/b/dave_froslie/archive/2016/02/18/automated-regression-test-qualities.aspx
- https://blogs.msdn.com/b/dave_froslie/archive/2016/02/03/achieving-balance-in-testing-software.aspx
- https://blogs.msdn.com/b/dave_froslie/archive/2011/08/19/are-all-systest-tests-unit-tests.aspx
- https://blogs.msdn.com/b/dave_froslie/archive/2011/07/26/using-the-systest-unit-testing-framework-in-dynamics-ax-why-bother.aspx
- https://blogs.msdn.microsoft.com/dave_froslie/2016/07/06/data-rollback/
- https://blogs.msdn.microsoft.com/dave_froslie/2016/03/09/test-automation-pyramid/
X++ test attributes
| Attribute Name | Usage |
| SysTestMethod | A method decorated with this attribute in a Test Class is considered as Test method (you can also just start the method name with 'test' and it will be treated as a test method. |
| SysTestInactiveTest | Deactivates the Test From Execution List |
| SysTestCaseConfigurationKeyConstraint | Class level and Method Level. The test can only run if the specified configuration key is enabled. It reports as inactive otherwise.example:[SysTestCaseConfigurationKeyConstraintAttribute(configurationkeystr(PMF)] //test will only run if the PMF configuration key is turned on |
| SysTestCaseConfigurationKeyDependency | Class level only. Requests the framework to activate/deactivate a configuration key temporarily for the purpose of the test.example:[SysTestCaseConfigurationKeyDependencyAttribute(configurationkeystr(PMF))] //test will always run. Turns PMF on during the test execution (if was off to begin with) [SysTestCaseConfigurationKeyDependencyAttribute(configurationkeystr(PMF), false)] //test will always run. Turns PMF off during the test execution (if was on to begin with) |
| SysTestCaseCountryRegionDependencyAttribute | Class level only. Requests the framework to run the tests in the provided country/region context.Example:[SysTestCaseCountryRegionDependencyAttribute('US')]This will enable US Country/Region features like tables, forms, etc. during test execution. It will also make country/region specific code execute. Please note that this feature works by injecting the country/region ISO code into the global cache. If for some reason the cache is cleared during test execution the effect dissapears. |
| SysTestCaseDependsOnReportAttribute | Method level only. Deploys the Report Design or all the designs within the Report.SysTestCaseDependsOnReportAttribute("BIR_RptNoParameter") // Only specify the report name without the design name in order to deploy all designs within a report. SysTestCaseDependsOnReportAttribute ( "BIR_SimpleReport.AutoDesign1" ) // Specifiy the report + design name to deploy a specific report design. |
| SysTestGranularity Parameters: | Marks the testclass or test method with this attribute Unit, Component, Integration, or BusinessCycle |
| SysTestFixture Parameters:[classstr(SysTestSuiteClass)] | The suite has the knowledge of what test classes or test methods but it’s not the other way. Where test classes or test methods know which suite they belong to. In Rainier, using the SysTestFixture Attribute, [SysTestFixture(classstr(WorkflowRuntimeTestSuite))]class SysWorkflowTest extends SysTestCase{public void testcancel() {}}public class WorkflowRuntimeTestSuite extends SysTestSuite{public void setUp(){}public void tearDown(){}}Through the SysTestFixture Attribute, we could enable test classes/methods to have the knowledge of which test suite they belong to. While executing the tests from visual studio tools (IDE and vstest.console.exe) SysTestFixture metadata is used to group the testmethods and execute them together, which would in turn run the suite-level setup() and teardown() once. |
| SysTestCaseNumSeqTypeDependency | Flags that the specific test class depends on a number sequence module. The system will ensure that the number sequence module and the specific data type is created prior to the test execution. Only the specified data type is setup for the execution of the test.Note: try to use SysTestCaseAutomaticNumberSequencesAttribute first before using this attribute. With this attribute the test case should upgrade code like this:SysTestCreateData_Basic::createNumberSequenceETId(NumberSeqModule::Invent, extendedtypenum(InventDimId));To an attribute like this:[SysTestCaseNumSeqTypeDependencyAttribute (extendedTypeStr(InventDimId), classstr(NumberSeqModuleInventory))]For tests that depend on a small set of number sequences, this is the preferred way of setting up number sequences when the test case is not compatible with SysTestCaseAutomaticNumberSequencesAttribute.The number sequences created with this attribute are based on the default settings of the data types.Any customization to the setup, format or parameters of the number sequence (i.e. "Continuous", "Manual", etc) needs to be applied during the SetupTestCase.Use of this attribute gives a very big performance gain compared to traditional methods such asSysTestCreateData_Basic::createNumberSequenceETId(NumberSeqModule::Invent, extendedtypenum(InventDimId)); |
| SysTestCaseNumSeqModuleDependency | Flags that the specific test class depends on all number sequences within a given module. The system will ensure that the number sequence module and all its data types are created prior to the test execution.Note: try to use SysTestCaseAutomaticNumberSequencesAttribute first before using this attribute. With this attribute the test case should upgrade code like this:SysUnitTestData_Proj::setupNumberSequences(NumberSeqModule::Sales);To an attribute like this:[SysTestCaseNumSeqModuleDependency(classstr(NumberSeqModuleSales))] For tests that depend on a many number sequences of a given module, this is the preferred way of setting up number sequences when the test case is not compatible with SysTestCaseAutomaticNumberSequencesAttribute.The number sequences created with this attribute are based on the default settings of the data types.Any customization to the setup, format or parameters of the number sequence (i.e. "Continuous", "Manual", etc) needs to be applied during the SetupTestCase.Use of this attribute gives a very big performance gain compared to traditional methods such asSysUnitTestData_Proj::setupNumberSequences(NumberSeqModule::Sales); |
| SysTestCaseAutomaticNumberSequencesAttribute | Flags that the specific test class may use number sequences and the system should create, at runtime and upon request, the right setup.With this attribute the test case should be removed of code similar to:SysUnitTestData_Proj::setupNumberSequences(NumberSeqModule::Sales);OrSysTestCreateData_Basic::createNumberSequenceETId(NumberSeqModule::Invent, extendedtypenum(InventDimId)); The use of this attribute actually changes the runtime behavior of number sequences to setup the data types and modules as they are requested.There is no performance cost of this attribute against the SysTestCaseNumberSeqModuleDependency or SysTestCaseNumberSeqTypeDependency and this mechanism ensures that, at runtime, only the needed types are created and no excessive setup is performed.This attribute may not produce the right results when the test case executes one of the following type of validation:· Tests that validate the number sequence framework itself.· Tests that expect non-existent number sequences.· A small number of tests validate number sequence reservation ("continuous") behavior may fail. The number sequences created with this attribute are based on the default settings of the data types.Any customization to the setup, format or parameters of the number sequence (i.e. "Continuous", "Manual", etc) needs to be applied during the SetupTestCase.Use of this attribute gives a very big performance gain compared to traditional methods such asSysUnitTestData_Proj::setupNumberSequences(NumberSeqModule::Sales); This attribute may produce a negative performance effect when you are replacing a very well behaved class that does one-time setup of one number sequence code. In this case, try using SysTestCaseNumSeqTypeDependencyAttribute (see above). |
| SysTestCaseDatasetDependency | Class level attribute. DOES NOT work when applied at method levelINHERITED (but not overridable) Flags that the specified test class needs to run on a given Dataset, starting on the specified company.Usage:SysTestCaseDatasetDependencyAttribute( str _datasetName, selectableDataArea _dataAreaId = 'DAT' )A dataset is a logical name of the Test data collection. It is not a partition where the data got imported to. The system will switch execution to the partition where the data is imported to and launch the execution on the company. If the dataset requested is not present in the system the test case will fail. |
| SysTestSecurityAttribute | Class level attribute. DOES NOT work when applied at method levelNOT INHERITEDBy default all test run with System administrator rights.In this example 2 personas are defined on a test class· each is given a different security role (the roles are defined in a list of assigned roles).· SystemUser role is automatically added by the framework· Note the TestEssentialsRole was explicitly added by the owner of the test since SystemUser does not have rights to all test framework endpoints. Task 1737753 will look into adding that role implicitly.· The old notion of a “user” is now called a “persona”, this is because we are not actually switching users midstream, but instead, are reconfiguring the user’s permissions· No kernel hooks were added for any of this, it only uses the development hook which was already available in 6.x.—‘SysAdminMode’· One of them sets default = true which means that the role becomes the default for the test methods (the setup() methods are all executed without a user context)· Using statements are not required, but are a preferred way of scoping the user’s context· The code controlling the behavior is completely encapsulated in the SysTestSecurityContext class.· MethodEnter/Exit and TestExecutable Enter/Exit control when settings are automatically changed back to admin and the default contexts· TestEssentialsRole is required for execution of form adaptor based testsSample: for complete reference see the SysTestSecurityContextTest class#define.PersonaSecRole('u1')#define.PersonaSecUserRole('u2')[SysTestTargetAttribute(classstr(SysTestSecurityContext), UtilElementType::Class),SysTestSecurity(#PersonaSecRole, [ 'SysTestSecurityContextTestRoleSecRole', 'TestEssentialsRole' ], true),SysTestSecurity(#PersonaSecUserRole, [ 'SysTestSecurityContextTestRoleSecUsrRole', 'TestEssentialsRole' ], false)]class SysTestSecurityContextTest extends SysTestCase-------NOTE---------------This attribute does apply the role correctly, but the priviliges assigned to any duty connected to this role will drive the security context. What it means is that if your Test is using a security role say HcmHumanResourceManager, which has one of the duties as HcmWorkforceProcessInquire which ultimately has the privilige HcmADARequirementView. This privilige has the entry point set to HcmADARequirement with a read access to display menuitem HcmADARequirement. So if you are using this SysTestSecurityAttribute with the HcmHumanResourceManager role, you will get view access if you do use form interaction (form adaptor based tests).But if you decide to directly interact with the table HcmADARequirement, you will end up getting a Delete access by default. If you don't want to test using the form adaptors but directly interact with the table (for eg) and still want the security context applied you will have to manually push the security context in your test code.SecurityContext context = SecurityContext::constructFromEntryPoint(SecurableType::MenuItemDisplay,menuItemDisplayStr(HcmADARequirement),''); context.push(); //Your code to ineract with the table goes hereSecurityContext ::pop(); |
| SysTestTransaction | Class level attribute. DOES NOT work when applied at method levelNOT INHERITEDDefines what is the test data rollback mechanism once the test class finishes.Default - [SysTestTransaction(TestTransactionMode::AutoRollback)]Uses the savepoint mechanism. Data is never committed to the database and is only accessible within the transaction scope. This was introduced with AX7.[SysTestTransaction(TestTransactionMode::LegacyRollback)]This is the same way as 6.0 and it saves the data in the database but tracks what records have been inserted and reverts them once the test execution is over.[SysTestTransaction(TestTransactionMode::None)]This will commit the data to the database and won't cleanup after the execution completes. |
How to generate form adaptors for new forms-
- Pre-requisite -> You have new module or model created for production code and also one for test code. Also at least one form is present in the model
- Open the descriptor file for the test module from the packages eg - I:\AOSService\PackagesLocalDirectory\NewTestModel\Descriptor\NewTestModel.xml and add this line
<FormAdaptorSourceModel>NewFormModel</FormAdaptorSourceModel>
This will mark this test module to hold the form adaptors for the NewFormModel model
- Open the properties window for the NewFormModel project and set the 'Generate form adaptors' property to true
- Now build the NewFormModel project and it will generate a new class called SomeSampleFormFormAdaptor under the NewTestModel. Refresh the application explorer and you should see the file.
How to record validation steps in task recording-
You can add validation steps to your task recording that will be converted to assert statements when you import the recording back into Visual Studio as a test.
There are two type of validations you can record
- Validate the value you see directly - Right click on the field you want to validate -> Task recorder -> Validate -> Current Value. This will always validate the value Contoso Europe in the example below and even the test generated from the recording will be validating this value.
- Validate the value by matching it with some other field - This is useful when you are actually less worried about what the value is, but in fact you want to make sure that the value matches some field that you copied earlier. From the example above, instead of clicking on Validate, you can click on Copy. Then on the Customer details form you can right click on the Name -> Task recorder -> Validate -> (Now you see two options). Instead of choosing Current value, you can choose to match the Name to the CustTable_Name field from the List page that you had copied. This is very use full if you want to track the progress of any field value.
Known Limitations
These are not necessarily limitations, but the current design.
- Tests can run in a clean partition if no dataset is provided or in the company belonging to the dataset that is loaded. You cannot combine tests that target different datasets because we don't have the capability to switch dataset on the fly. It will be costly to switch as well since it takes a long time to drop and restore the database. If you want to have tests that target different sets of data, then prepare your database with different partition and that way the partition can be switched easily to execute the tests.
- Data created during a test run gets cleaned up after the test finishes. Currently we don't support running the test in any particular order. So you cannot assume that the data from test1 can be used by test2.