MSTest is a good unit test solution , tightly integrated with Visual Studio IDE .Following is a typical flow for test automation development and running the tests
- Tests are developed and tested completely in VS IDE .
- MSTest.exe is used for running the tests in lab .
MSTest.exe is the command line test execution tool . Details about various options that it provides are here .
Mostly , MSTest.exe and VS IDE are similar . Following are some subtle differences
- If a project has other test projects referenced , VS IDE would run tests from all the referenced projects. Whereas MSTest.exe would only execute the tests that are directly in the dll provided in command line for /testcontainer parameter.
- VS IDE finds out all the necessary dlls required , it walks the depency graph and does this for us. MSTest.exe doesn’t do this . This might result in a failure during automated runs,especially when the tests are run in remote machine using a Test Controller + Test Agent setup . Following are some solutions to this issue
- Make sure “Copy Local” option is set for all custom dlls that are referenced.
- Use Deployment Item option in TestSettings file to copy dlls that will not be available in the machine where tests will be run. Pass this test setting file as a parameter to MSTest.exe command line.
- VS IDE works with relative paths .MSTest.exe won’t in most cases , because design time relative path and run time relative paths would most likely be different. This issue shows up mostly for load tests.Load tests are xml files , Unit tests that are part of loadtests could be specified using relative path or using absolute path. While developing the load test with VS IDE , it adds the tests using relative path. This may not work while running the load test from command line using MSTest.exe . Following are some possible solutions for this issue
- Provide a placeholder location during build time (like an environment variable) , and replace it during automation run time. For example the location could be “%BuildRoot%\TestDll.dll” at design time and changed to “c:\CurrentBuild\TestDll.dll” at run time.