Visual Studio Team System 2005 Code Coverage Tool FAQ

This FAQ will collect the most common questions that are asked about the code coverage tool that is included with Visual Studio Team System for Developers (and Team Suite edition).

Q: Why am I seeing some uncovered blocks in my program when I’m sure that my test should be exercising 100% of my code?

A: The simple answer is that while you may be sure that you are testing 100% of the visible lines of code there are a few small cases where there are underlying IL blocks that are not being tested. It’s important to know that (in all the cases that we’ve found so far) the code coverage tool is not lying to you; there are code coverage blocks that were not covered by the test. Usually, this is cause by some extra lines of IL that the CLR adds in under the covers.

A few of the times that we have already seen these problems are with foreach loops and with switch statements that check six or more string values. If you see any more examples of this behavior please report them on the forum.

Q: I’m seeing partially covered lines in a statement with logical operators (||, &&, ect) what does a partially covered line mean?

A: The code coverage tool operates on blocks as opposed to lines of code. For example, say that you are testing the following statement:

if(A || B)

If you run a test with code coverage with the statement A evaluating to true you will see the above line as a partially covered line. Since A was true the B block was not evaluated so it was not covered by code coverage. Since A was covered but B was not this shows up as a partially covered line. To get full coverage you will have to modify you test to make sure that the B statement is also covered.

Q: I already have some unit tests for my project. How to I collect code coverage information from them?

A: Navigate to the code coverage tab in your current test run config. In that page, select all of the binaries that you want to collect code coverage from. Note that for any signed binaries you will need to click the re-sign button to resign them after they are instrumented for code coverage. Then just run your tests as normal. After a test run just right click on the test run results and select “Code Coverage Results” to view the code coverage data. For a more detailed look at this process take a look at the following TechNote:

http://msdn.microsoft.com/vstudio/teamsystem/reference/technotes/code_coverage/basics_ide.aspx

Q: How do I merge the results from multiple code coverage runs?

A: In the code coverage results window is a small button with a green plus sign placed between two white boxes. When you click this button you will see the results from the last group of test runs. To join code coverage information, just select all the runs you want to join and click ok to create a set of merged results.

Q: How can I collect and manipulate code coverage data programmatically?

A: We know that many users will be doing code coverage in some sort of custom build scenario and won’t be able to use the excellent IDE integration, so we provided tools to collect and manipulate code coverage data programmatically. This topic is a little too broad for a FAQ entry and is best covered by Joc’s excellent article on the topic, linked below:

http://blogs.msdn.com/ms_joc/articles/406608.aspx

Q: Can I collect code coverage data from web tests?

A: At this time we do not support collecting code coverage information from web tests. You can collect code coverage information from unit tests and other test types for which you have symbols and where you can select what binaries to instrument (via the test run config file).

Q: What are the basic steps for using the command line tools to collect code coverage data?

A: You will need to use vsinstr and vsperfcmd tools that are found in the Team Tools/Performance Tools directory under your Visual Studio installation directory. For the binary that you are going to test run the following vsinstr command line to instrument the binary for code coverage:

>vsinstr –coverage MyAssembly.exe

Then you need to start up the monitor to collect the data. You do this with the vsperfcmd tool:

>start vsperfmon –coverage –output:mytestrun.coverage

Now run your tests or exercise the code that you want to collect code coverage data for. When your program finishes, you will have a .coverage file that you can open in the IDE for analysis.