Live Unit Testing in Visual Studio 2017 RC

We are very proud to introduce a new feature in Visual Studio 2017 called Live Unit Testing ! This feature will make it easy for you to maintain quality and test coverage during rapid development and take your productivity to a whole new level. Imagine you are fixing a bug in a code base which you may not be completely familiar with. With Live Unit Testing you can know right away, as you are making edits to fix the bug, that you did not break any other part of the system. Getting this feedback, in-situ, as you type will give you extra confidence, make you more productive and why not, even enjoying fixing bugs and writing unit tests!

Live Unit Testing automatically runs the impacted unit tests in the background as you edit code, and visualizes the results and code coverage live, in the editor, in real-time. In addition to giving feedback on the impact that your changes had on the existing tests, you also get immediate feedback on whether the new code you added is already covered by one or more existing tests. This will gently remind you to write unit tests as you are making bug fixes or adding features. You will be on your way to the promised land where there is no test debt in your code base!

Live Unit Testing is present in the Enterprise edition of Visual Studio 2017 and it’s available for C# and VB projects that target the .NET Framework. It uses VB and C# compilers to instrument the code at compile time. Next, it runs unit tests on the instrumented code to generate data which it analyzes to understand which tests are covering which lines of code. It then uses this data to run just those tests that were impacted by the given edit providing immediate feedback on the results in the editor itself. As more edits are made or more tests are added or removed, it continuously updates the data which is used to identify the impacted tests.

How to start Live Unit Testing

Enabling Live Unit Testing is as simple as going to the Test command at the top level menu bar and starting it as shown in image below.

Live Unit Testing in Visual Studio works with three popular unit testing frameworks; namely, MSTest, xUnit and NUnit. When using these, you will need to ensure that the adapters and frameworks meet or exceed the minimum versions given below. Please remove older adapter and test framework references from your existing projects ( make sure you remove the reference to “Microsoft.VisualStudio.QualityTools.UnitTestFramework”) and add the new ones if Live Unit Testing is not working for you. You can get all of these from NuGet.org.

  • For xUnit you will need xunit.runner.visualstudio version 2.2.0-beta3-build1187 and xunit 2.0 (or higher versions)
  • For NUnit you will need NUnit3TestAdapter version 3.5.1 and NUnit version 3.5.0 (or higher versions)
  • For MSTest you will need MSTest.TestAdapter 1.1.4-preview and MSTest.TestFramework 1.0.5-preview (or higher versions)

Live Unit Testing experience

Once enabled, Live Unit Testing helps you quickly see whether the code you’re writing is covered and if the tests that cover it are passing, without leaving the editor. Unit test results and coverage visualizations appear on a line-by-line basis in the code editor as shown in sample image below:

clip_image001 If a line of executable code is covered by at least one failing test, Live Unit Testing will decorate it with a red “×”.

clip_image002 If a line of executable code is covered by only passing tests, Live Unit Testing will decorate it with a green “✓”.

clip_image003 If a line of executable code is not covered by any test, Live Unit Testing will decorate it with a blue dash.

The real-time code coverage and test result information provided by Live Unit Testing removes the burden of manually selecting and running tests. The live feedback also serves to notify you instantly if your change has broken the program – if inline visualizations shift from green “✓”s to red “×”s, you know you broke one or more tests.

At any point in time you can hover over the “✓” or “×” to see how many tests are hitting the given line as seen in image below.

You can click on the check or “×” to see what tests are hitting the given line as shown in image below.

When hovering over the failed test in the tool tip, it expands to provide additional info to give more insight into the failure as shown in image below.

Additionally, you can navigate directly to the failed test by clicking on it in the tool tip. Then, from the failed test, you can easily debug to the product code, make edits, and continue, all while Live Unit Testing runs in the background. There is no need to stop and restart Live Unit Testing for the debug, edit and continue cycle.

At any time, you can temporarily pause or completely stop live unit testing; for example, when you are in the middle of a refactoring and you know that your tests will be broken for a while. It is as simple as going to the Test command in top level menu bar and clicking the desired action, as shown below.

When paused, you will not see any coverage visualization in the editor. When you are ready to see them again, you can un-pause it by clicking “Continue” in the Live Unit Testing menu. When in pause mode, Live Unit Testing still keeps all the data that it had collected thus far. On “Continue,” Live Unit Testing will do the necessary work to catch up with all the edits that have been made while it was paused, and will update the glyphs appropriately.

You can also stop Live Unit Testing completely if you desire. When Live Unit Testing is started again after it has been stopped, it will take longer to show the glyphs than when it was un-paused. This is because it loses all data when it is stopped.

Conclusion

Live Unit Testing will improve your developer productivity, test coverage and quality of software. .NET developers out there, please do check out this feature in Visual Studio 2017. For developers who are part of a team that practices test-driven development, Live Unit Testing gamifies their workflow; in other words, all their tests will be failing at first, and as they implement each method they will see them turn green. It will evoke the same feeling as a nod of approval from your coach, who is watching you intently from the sidelines, as you practice your art!

For problems, let us know via the Report a Problem option in the upper right corner, either from the VS installer or the Visual Studio IDE itself. If you’d like to leave us a suggestion then please use User Voice

Watch this Live Unit Testing video, where we demonstrate this feature.

Joe Morris, Senior Program Manager, Visual Studio
@_jomorrisJoe has been with Microsoft for 19 years, with a particular focus on static analysis and developer productivity for the last three years.
Manish Jayaswal, Principal Engineering Manager, Visual Studio

Manish has years of management experience in commercial software development with deep technical expertise in compiler technology, debugger, programming languages, quality assurance and engineering system.