Code Coverage using Visual Studio 2008’s Code Coverage tools

  • Article

The code coverage tools in Visual Studio have existed since 2005 in the Team Developer and Tester Tools -> Code Analysis Tools in Visual Studio Team System 2008 installation. However, there have not been enough articles about how to use them on Windows Service and WCF (Windows Communication Foundation) applications. The steps are not as easy as using the Test Run Configuration’s Code Coverage at https://msdn.microsoft.com/en-us/library/ms182496.aspx. You actually need to run the tools at command line to start and stop the code coverage monitor. I talked to the engineers in the Visual Studio team and compiled the official steps for doing code coverage on Windows Service and WCF applications. The biggest problem of the code coverage tools is that it does not support 64-bit code coverage collection. The reason is that there is no integrated solution for 64-bit code coverage collection in Test Run Configuration’s Code Coverage. Visual Studio team decided to proactively deny instrumenting 64-bit assemblies for code coverage in Visual Studio 2008. The tool executables can be found at \Program Files (x86)\Microsoft Visual Studio 9.0\Team Tools\Performance Tools.

WCF application code coverage:

 

  1. Instrument the web service binary that you want coverage on: VsInstr.exe /coverage myWebService.dll. Deploy the dll to IIS 6 or 7.
  2. Start the monitor for collecting coverage data: VSPerfCmd.exe /start:coverage /output:”C:\CoverageData.coverage” /cs /user:”NT AUTHORITY\Network Service” /user should be whatever identity IIS is running under. /cs means cross session which allows the code coverage collection to be run from other sessions such as making a WCF call from a remote machine.
  3. Run the scenario or test cases that will exercise the WCF service.
  4. Verify the monitor has started: VSPerfCmd.exe /status: PS D:\Users\hiliao> & 'D:\Program Files\VSCC\VSPerfCmd.exe' /status

Microsoft (R) VSPerf Command Version 9.0.30305 x64

Copyright (C) Microsoft Corp. All rights reserved.

Process and Thread Status

============================================================

Output File Name : D:\tmp\CoverageData.coverage

Collection mode : COVERAGE

Maximum Processes : 64

Maximum Threads : 256

Number of Buffers : 258

Size of Buffers : 65536

============================================================

Maximum Number of Processes : 64

Number of Active Processes : 1

Global Start/Stop Count : 1

Global Suspend/Resume Count : 0

------------------------------------------------------------

 Process : d:\windows\system32\inetsrv\w3wp.exe

 Process ID : 6604

 Num Threads : 0

 Start/Stop Count : 1

 Suspend/Resume Count : 0

============================================================

Users with access rights to monitor:

UserName (SID)

NT AUTHORITY\NETWORK SERVICE (S-1-5-20)

BUILTIN\Administrators (S-1-5-32-544)

NT AUTHORITY\SYSTEM (S-1-5-18)

PS D:\Users\hiliao> 

  Shutdown the monitor when your scenario is done: VSPerfCmd.exe /shutdown. You will see on the console: Waiting for process 6604 ( d:\windows\system32\inetsrv\w3wp.exe) to shutdown...

 

Windows Service code coverage:

 

  1. Stop the service you want to test against 
  2. Start the monitor at Powershell prompt: & 'C:\Program Files\VSCC 10.0\VSPerfCmd.exe' /start:coverage /output:"D:\Latest.Coverage" /cs /user:"Everyone" ; change “Latest.Coverage” to whatever filename you want for the coverage file. Keep the “.coverage” filename extension.
  3. Start the services you stopped
  4. Check if the instrumented files are being monitored. Run at Powershell: & "\Program Files\VSCC 10.0\VSPerfCmd.exe" /status.

 

Microsoft (R) VSPerf Command Version 9.0.30305 x64

 

 

Copyright (C) Microsoft Corp. All rights reserved.

Process and Thread Status

============================================================

Output File Name : E:\build73FST2.coverage

Collection mode : COVERAGE

Maximum Processes : 64

Maximum Threads : 256

Number of Buffers : 258

Size of Buffers : 65536

============================================================

Maximum Number of Processes : 64

Number of Active Processes : 1

Global Start/Stop Count : 1

Global Suspend/Resume Count : 0

------------------------------------------------------------

 Process : D:\Program Files\Product\PRODUCT Bulk Service\ProductBulkService.exe

 Process ID : 6768

 Num Threads : 0

 Start/Stop Count : 1

 Suspend/Resume Count : 0

============================================================

Users with access rights to monitor:

UserName (SID)

\Everyone (S-1-1-0)

BUILTIN\Administrators (S-1-5-32-544)

NT AUTHORITY\SYSTEM (S-1-5-18)

If you can’t get any code coverage results, look at application event log (eventvwr.exe) with Source == VSPERF. There might be useful information about why the monitor is not running.

 

You can probably find x64 folder in \Program Files (x86)\Microsoft Visual Studio 9.0\Team Tools\Performance Tools. However, a necessary component VSCover90i.dll on 64 bit Windows is missing which causes collecting 64 bit code coverage to fail. The output message is from running the internal code coverage tools build at Microsoft. I am hoping the next release of Visual Studio will have the necessary DLL to run 64 bit code coverage.

 

  1. Run the scenario or test cases that will exercise the Windows Service
  2. Stop the Windows Service and Shutdown the monitor: VSPerfCmd.exe /shutdown. The code coverage data should be saved at C:\CoverageData.Coverage. Do not move the .pdb files of the instrumented EXE or DLL. Otherwise, C:\CoverageData.Coverage will fail to load.

 

Instrument the Windows Service executable that you want coverage on: VsInstr.exe /coverage WinSvc.exe. Deploy the exe to the Windows Service folder.

 

  1. Start the monitor for collecting coverage data: VSPerfCmd.exe /start:coverage /output:”C:\CoverageData.coverage” /cs /user:”Everyone”
  2. Start the Windows Service.
  3. Verify the monitor has started: VSPerfCmd.exe /status. You will see on the console: PS D:\Users\hiliao> & 'D:\Program Files\VSCC\VSPerfCmd.exe' /status

 

1.

  • The command does not return until you totally stop IIS for process w3wp.exe to exit. When the command returns. The code coverage data should be saved at C:\CoverageData.Coverage. Do not move the .pdb files of the instrumented EXE or DLL. Otherwise, C:\CoverageData.Coverage will fail to load.