The TestContainer Capability
Updating off pre-RTM bits once RTM ships ought to be routine. But if you have not already done so in the case of the .NET Core based Test projects, let me give you a reason to do so.
vstest delegates discovery and execution of tests to test-framework-specific adapters. Adapters indicate the kind of test containers that they can process – for e.g. in the case of .NET based test frameworks like MSTest, xUnit, NUnit, etc. their adapters would indicate that they can discover and execute tests from “.DLL” files. Now imagine a solution producing say 20 DLLs, and having test projects that use xUnit (or NUnit, or MSTest). When it comes time to discover tests, the Visual Studio Test Explorer hands off all 20 DLLs to the adapter. But say only 2 of those DLLs contained tests while the other 18 DLLs were product code. The adapter ends up rejecting those 18 DLLs, and processing only the 2 DLLs. Due to this activity with the non-tests DLLs, there is a performance hit, and there are avoidable diagnostics spewed out. Had there been other adapters that had indicated that they could process “.DLL” files, the 20 DLLs would have been handed off those adapters as well.
An optimization in the Visual Studio Test Explorer
What is needed is distinguishing DLLs capable of containing tests.
Which brings us to “capabilities”. Project capabilities – documented here – are the recommended way to determine the type, platform, and features of a project. We have started leveraging this for .NET Core Test projects. .NET Core Test projects referencing the Test SDK v15.0.0 and above have a capability called “TestContainer” that distinguishes them as a test project. Issue #440 on GitHub traces the details for this feature.
Starting with Visual Studio 15.3 Preview 4, the Test Explorer hands off only the projects that have this capability to the relevant adapters. Going by the earlier example, only the 2 test projects will be handed off to the adapters! This streamlines the experience from both the performance and diagnostics perspectives.
The .NET Core Test SDK went RTM with v15.0.0 (more than 6 months back).
.NET Core Test project referencing a pre-RTM version of the Test SDK do not have this capability, and test from such projects will not get discovered in the Test Explorer of Visual Studio 15.3 Preview 4 and above (there is no impact to CLI and CI scenarios). But this is easily addressed – on such old test projects, just upgrade your Test SDK reference to the latest!
There, now you have a reason to update.