Testing a Team Services extension and tracking code coverage

Jasmine is a framework that allows you to write tests in JavaScript for AngularJs or NodeJs applications. In our case, we will use Jasmine to write tests for VSTS extension.

Implementing Jasmine on an Extension Project

The Jasmine Framework is a Jasmine.js library. To retrieve it, we have two possibilities, either with a package Nuget or with an npm package.
In case the extension is developed with Visual Studio, we retrieve the Chutzpah package with Nuget that contains jasmine and its test adapter.


With Visual Studio Code (or with another IDE), we can retrieve this framework with npm with the command.

npm install --save-dev jasmine

If you are using Josh Garverick’s Visual Studio Project Templates, all components required to run Jasmine tests are already included in the npm packages configured in the "packages.json" file.


Write your unit tests for your extension

Writing a Jasmine unit test is fairly simple. The Project Template has an example in the test folder in the "testSpec.js" file.

Here is another example:


Run your tests in Visual Studio or Visual Studio Code

In Visual Studio

There are several ways to run Jasmine unit tests. With Visual Studio, you can use a test adapter like Chutzpah, which is one of the best known.  It is installed with Nuget by retrieving the latest version of the Chutzpah package.

Next, to be able to visualize and execute the tests in the Visual Studio Test Explorer, install an extension to Visual Studio called Chutzpah Test Adapter for the Test Explorer.  Additionally, install the Chutzpah Test Runner Context Menu Extension to be able to execute tests from the contextual menu.


Once installed, the tests appear in the explorer test, and they can be run as unit tests.


In Visual Studio Code

You can use the grunt-contrib-jasmine grunt already existing in the template, and in the "gruntfile.js" file have the jasmine task.


Run grunt jasmine


Chutzpah file configuration : Chutzpah.json

To be able to set Chutzpah parameters, we add a configuration file json (at the root of the project) named "Chutzpah.json". Inside, we specify the file of specs to be executed, the mode of compilation of the typescript files, and the parameters of execution of the code coverage.

For all details on this file, see https://github.com/mmanela/chutzpah/wiki/Chutzpah.json-Settings-File

Running tests as part of Continuous Integration (CI)

IMPORTANT - The following section contains paths as part of the configuration. These are solution-specific and likely to differ in your environment. Use the screenshots and examples as reference only! 

To run the JavaScript tests in a build of Continuous Integration in VSTS, add the Test Assemblies task to your build definition.


Set parameters

  1. Execution Options - The tests path and pattern.
  2. Advanced Execution Options - The root of test adapter : the packages folder where the Nuggets packages, including the Chutzpah package, are downloaded.

After the build, the results of the tests are obtained in the Summary tab.


And the details on the tests tab.


Code coverage analyse

Enabled the code coverage

The code coverage indicators provide an indication (in percentage) of the number of lines covered by the tests.

One of the advantages of Chutzpah is that it allows this analysis and generates a report during the tests execution.
To activate it, add the parameters in the configuration file of chutzpah.


  1. Enable the code coverage analysis and the files to includes and excludes.
  2. Specify the file type for the analysis result; in our case, we will choose the lcov format.

Executing the code coverage analysis during CI build

With this configuration, the code coverage analysis will be done automatically at the same time that the tests will be executed, so there is nothing more to configure in the build.

After the build runs, the result of the analysis is visible in the summary tab.


Publish and view the detailed analysis report

Having this percentage is a good indication, but it's better to have a detailed report that indicates on each file the lines that are not covered by the tests in order to improve them.

In the result of the VSTS builds, the code coverage tab allows you to display this report, which is in the Jacoco or Cobertura format.

The problem we have is that the report generated by Chutzpah is in lcov format, so in the build execution convert this report into a report summary of type Cobertura, then, in html format, and, finally, publish this report.

  • The first step is to convert the file "lcov.dat" obtained during the execution of the tests in "Cobertura.xml" format. This conversion is done with a Python script that is available here.  Add this script into the repository, and execute it during the build with a Command Line task.


$(Build.SourcesDirectory)\CountdownWidget\lcov_cobertura.py  $(Build.SourcesDirectory)\CountdownWidget\CountdownWidget\lcov.dat --output $(Build.BinariesDirectory)\Cobertura.xml

  • It is necessary to generate a report in html format from the "Cobertura.xml" using the tool ReportGenerator which is obtained in a package Nuget (included in our project), available here.



-reports:$(Build.BinariesDirectory)\Cobertura.xml -targetdir:$(Build.BinariesDirectory)\reports -reporttypes:Html;Xml;XmlSummary

  • The last step is to publish the report generated by ReportGenerator by using the Publish Code coverage result task.




Once the build is running, you get the report in the Code Coverage tab.




  1. Add Jasmine to your project
  2. Add and validate unit tests
  3. Add and configure Test Assemblies task to your CI build
  4. Configure Chutzpah configuration to enable code coverage
  5. Publish code coverage results
Comments (9)

  1. There are easier ways to publish the code coverage report, since Chutzpah can also output in JaCoCo format and an HTML report, using the Transforms settings (it can output multiple formats at once). See here https://medium.com/nov-rsis-development/enabling-javascript-typescript-code-coverage-in-a-vsts-tfs-build-36fe41d40f97#.qe6ly49kq

    1. mikaelkrief says:

      Chutzpa provide xml summary JaCoCo code coverage but doesn’t provide the Html report Jacoco or Cobertura format for the Code coverage tab in the build, so I need to use Report Generator for tranform xml summary to html compatible report

  2. pabrams1 says:

    Any hints on using this with typescript (and webpack)? I tried adding
    `import {} from ‘jasmine’;`
    to the top of your example spec for adding two numbers, but when I do `npm run test` it says ‘define is not defined’:

    C:\git\myproject>npm test

    > myproject@0.0.1 test C:\git\myproject
    > jasmine-ts “src/spec/**/*.spec.ts”

    import {} from ‘jasmine’;
    ReferenceError: define is not defined
    at Object. (C:\git\vsts-team-calendar\src\spec\test.spec.ts:1:63)

    at Module._compile (module.js:635:30)
    at Module.m._compile (C:\git\myproject\node_modules\ts-node\src\ind
    at Module._extensions..js (module.js:646:10)
    at Object.require.extensions.(anonymous function) [as .ts] (C:\git\myproject\node_modules\ts-node\src\index.ts:395:12)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    npm ERR! Test failed. See above for more details.


    I’m also using webpack, not sure if I have to do something with that?

  3. pabrams1 says:

    Under the heading “In Visual Studio Code”, you say: “You can use the grunt-contrib-jasmine grunt already existing in the template, and in the “gruntfile.js” file have the jasmine task.”

    What project template? The only project template I see is the Visual Studio template you mentioned, but this isn’t Visual Studio, it’s VS Code.

    1. mikaelkrief says:

      The project template that I memtion is the VS project template for VSTS extension provided by Josh Garverick.
      In the VS code section I expose how run the js tests from VS Code, but you need to install the template in VS.

      However, I suggest to you also read this article https://blogs.msdn.microsoft.com/visualstudioalmrangers/2017/03/16/generate-team-services-extension-starter-kit-with-the-yeoman-generator-vsts-ext/ which is more recent about templates for VSTS extensions

      1. pabrams1 says:

        Thanks, any guidance on using this with typescript? Jasmine keeps saying “define is not defined”.

          1. Anonymous says:
            (The content was deleted per user request)
          2. Paul Abrams says:

            Hi again, thanks for the help – I tried to apply the same strategy used by countdown widget extension to my own project, but I get “cannot find module VSS/Utils/Date”. Would you know of any projects that are successfully unit testing modules that import VSS SDK components?

Skip to main content