Test Attributes and Filtering

By popular demand, Platform Update 12 contains improvements on SysTest class and method attributes in X++, and how they behave in the Visual Studio test window as well as the VS Test Console, which is the tool used in the automated build process. Our goal was to improve and support the major test attributes in our SysTest framework adapter to be on par with the MSTest framework adapter. This includes attributes like Category, Owner, Priority and Test Property.

TestCategory

The Category attribute (SysTestCategoryAttribute) was already available in previous releases using the SysTestCategory attribute. With the new platform update however, you are now able to specify multiple categories on both the class level and the individual method level. Additionally, the TestCategory is now enabled for filtering in the VS Test Console, which means you can create build definitions with test filters on specific categories. You can use the "TestFilter" variable in the build definition for this, for example set the variable to "TestCategory=Nightly" to only run tests with a category "Nightly".

Priority

The Priority attribute (SysTestPriorityAttribute) has been added, requiring an integer value. A priority can only be specified once but is supported on both class and method level, with method level taking precedence over class level. It is also exposed as a test filter so you can use the "TestFilter" variable in the build definition for this, for example set the variable to "Priority=1" to only run tests with a priority of 1.

Owner

The owner attribute (SysTestOwnerAttribute) has been added as well. This attribute was technically already supported for filtering in the Test toolbox window but the attribute itself was missing in X++. Like priority, an owner can only be specified once and is supported on both class and method level, with the method level taking precedence. Owner is not available as a test filter for the console, which is in line with the MSTest adapter for VS Test Console. Owner will show up in the Test toolbox window in Visual Studio, however.

Test Property

The test property attribute (SysTestPropertyAttribute) has existed in previous releases of the platform, but wasn't fully functional. Unfortunately the SysTestPropertyAttribute exists in the Application Foundation package as opposed to the other attributes which exist in Test Essentials. Due to our commitment to backwards compatibility of the platform we can't currently move this attribute to its expected location, so you will need to add a reference to the Application Foundation package to use it. Test Property specifies a property and a value (two strings) and can now be used in the Test toolbox window in Visual Studio. Test properties can be specified multiple times, and can exist on both class and method level. Test property is not available for test filtering in the VS Test Console, in line with the MSTest adapter.

For advanced filtering syntax that can be used with the VS Test Console, see Running selective unit tests on Microsoft Docs for .NET and review the filtering examples for the MSTest framework.

Note that for test filtering purposes we have also always allowed you to filter on FullyQualifiedName and Name, and those features continue to exist.