JavaScript Unit Test Intergration in VSTS Build Pipeline

VSTS supports any kinds of JavaScript test runner as long as it can be invoked via command line, we can either use Chutzpah with PhantomJS or karma with most popular browsers in its build pipeline. This article will demonstrate how to integrate karma unit test into VSTS and notice that any other test runner just can follow similar patterns to accomplish the integration.

Configure Unit Test

Execute the following commands to initialize the karma test environment:

 
npm init
npm install karma --save-dev
npm install jasmine-core --save-dev
npm install karma-junit-reporter --save-dev
npm install gulp --save-dev
karma init

The following is the generated karma.conf.js with a little bit code modification:

 
module.exports = function(config) {
 config.set({

  // base path that will be used to resolve all patterns (eg. files, exclude)
  basePath: '',

  // frameworks to use
  // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
  frameworks: ['jasmine'],

  // list of files / patterns to load in the browser
  files: [
    'specs/*.js'
  ],

  // list of files to exclude
    exclude: [
  ],

  // preprocess matching files before serving them to the browser
  // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
  preprocessors: {
  },

  // test results reporter to use
  // possible values: 'dots', 'progress'
  // available reporters: https://npmjs.org/browse/keyword/karma-reporter
  reporters: ['progress', 'junit'],

  // web server port
  port: 9876,

  // enable / disable colors in the output (reporters and logs)
  colors: true,

  // level of logging
  // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
  logLevel: config.LOG_INFO,

  // enable / disable watching file and executing tests whenever any file changes
  autoWatch: false,

  // start these browsers
  // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
  browsers: ['Chrome'],

  // Continuous Integration mode
  // if true, Karma captures browsers, runs the tests and exits
  singleRun: true,

  // Concurrency level
  // how many browser should be started simultaneous
  concurrency: Infinity
 })
}

Add test case to the spec folder and run "node .\node_modules\karma\bin\karma start" should see the test cases can be executed as expected by Chrome.

Integrate with Task Runner gulp

VSTS supports node command task while we can use gulp to drive the test task even easier in a more complex scenario. We can add gulpfile.js shown below to enable this functionality. Then, we should be able to run the test case by invoke command "gulp unittest".

 var gulp = require('gulp');
var Server = require('karma').Server;
gulp.task('unittest', function(done){
    new Server({
        configFile: __dirname + '/karma.conf.js',
        singleRun: true
    }, done).start();
});

Support Headless Testing

Before adding the build pipeline to VSTS, we need think about what build agent we will use to run the test case on. Apparently, the most convenient ways is to use hosted agents which are maintained by VSTS as we don't need to take care of the restart, patch anymore. While, the hosted agents don't have Chrome installed and the above test will always fail because of that. Alternatively, we can use private agent which are maintained by ourselves and can install any required software as we want, we just follow https://docs.microsoft.com/en-us/vsts/build-release/concepts/agents/agents to configure a private host and to VSTS. In this example, I will use a hosted VS2017 agent to make it. In order to run karma on a hosted agent without Chrome installed, we need support headless testing:

• Run "npm install puppeteer --save-dev"
• Add "process.env.CHROME_BIN = require('puppeteer').executablePath()" to the beginning of karma.conf.js
• Change browsers setting from Chrome to ChromeHeadless

Setup Continuous Integration

In VSTS Build & Release tab, add a new build definition. Then add a "npm install" task shown below. WebApp/JSUnitTest is the working folder that contains the project.json.

Add a gulp task shown below, make sure the gulp task name matches the task name defined in the gulpfile.js.

In Triggers tab, enable trigger shown below. Then the CI build process will automatically run the test cases every time a team member commits changes to version control.

Add JUnit Test Report

Now any code change to the repository will trigger a new build process, you might want to check the test result instead of a rough test succeed or fail result. Currently, the karma.conf.js has been able to generate junit result by reporters setting. We just need to  enable JUnit Test Results  in gulp task shown below to let VSTS understand where to grab the test result.

Once a build completes, select the build number and we can see the test result in summary.