New unit testing platform in Visual Studio 2012 provides a new way to selectively execute test based on filtering condition through TestCaseFilter. TestCaseFilter can as specified as a string while executing tests through command line (vstest.console.exe), Team Build (when running test using “Visual Studio Test Runner”) or through Test platform client API.
Support for adapters to leverage this feature is provided in test platform. MSTest adapter shipped with Visual Studio 2012 for executing managed test provide basic filtering as described below (more enhanced filtering expression support to be added in future versions).
- Command Line
New command line runner for executing tests in VS 2012 (vstest.console.exe) has an optional command line parameter /TestCaseFilter for specifying filtering expression. /TestCaseFilter cannot be specified with /Tests argument.
E.g. vstest.console.exe test.dll /TestCaseFilter:”TestCategory=Nightly”
- Team Build
When adding a test activity in build definition, if user selects to execute test using “Visual Studio Test Runner” then user get to specify test case filter (optional can be left blank to run all tests).
- Client API
When using Test Platform Client API for executing test then filtering expression can be specified in TestRunCriteria.TestCaseFilter property.
TestRunCriteria runCriteria = CreateTestRunCriteria();
runCriteria.TestCaseFilter = “TestCategory=Nightly”;
Syntax for filtering expression:
Adapters can choose their own syntax for filtering or take advantage of support provided by Test Platform by MSTest adapter syntax. Following is syntax for MSTest adapter for managed test execution. Invalid filtering expressions will be ignored and all tests will be executed.
Operators supported in RC are:
- = (equals)
- != (not equals)
- ~ (contains or substring only for string values)
- & (and)
- | (or)
- ( ) (paranthesis for grouping)
Expresssion can be created using these operators as any valid logical condition. & (and) has higher precedence over | (or) while evaluating expression.
"Priority~1" // Invalid as priority is int not string
Properties supported by MSTest adapter for filtering are
- ClassName=<ClassName> (Valid only for unit tests for Windows store apps, currently not available for classic MSTest)
Using TestCaseFilter in Test Adapters
Adapters can leverage default format of TestCaseFilter that MS Test uses. Parsing and matching is done by ITestCaseFilterExpression provided by test platform.
Following method from IRunContext is for getting an implementation of ITestCaseFilterExpression.
ITestCaseFilterExpression provides following method to match test case (only if MSTest format is used for TestCaseFilter)