Running Windows 8 Metro style tests in TeamBuild

Visual Studio 11 Beta was announced a week back along with Win 8 consumer preview.

Unit testing has been redefined in a more framework agnostic way in Visual Studio 11. Refer our MSDN articles and introductory post to get started with it and expect more posts from us in the coming weeks.

Today, let’s quickly go though the steps to get a simple metro style test running in your nightly build.

Installing and configuring the build agent:

  1. Install the Team foundation Build agent on the Windows 8 box.
  2. Configure the build agent to run as interactive process. This is needed since metro style apps could not be launched from a background service.
    1. Enable auto-logon for the build agent service account [so that the build machine always has an active logged in session].
    2. Make sure that the build agent service account has permissions to write to the build drop and it is a member of “ProjectCollection Build service Account group”
  3. Install Visual Studio Win8 express on the build agent machine. This is needed to bring in the Windows 8 specific dependencies.
  4. Install developer license in the build agent box. This is needed to run metro style tests.

Authoring the tests:

  1. Create a simple C# metro app and unit tests for it.
  2. Launch VS. While choosing project template, select "Window metro style" and chose the appropriate project type.
  3. Author the tests and build the project. More details can be found on MSDN.
  4. On a successful build, "Unit test explorer" window would open on the left pane showing the test run progress.

Creating build definition:

            Couple of things to take care of while specifying test parameters in the build definition:

    1. Specify "Visual Studio Test Runner” in the TestRunner dropdown. This would be the default value.This ensures that the tests are run via the new Visual Studio 11 based test runner instead of MSTEST.exe.
    2. Specify *.appx for test assembly specification instead of .dll. For running metro style tests .appx file has to be given as input to the test runner instead of .dll.


Installing the certificate:

Just one more step to go before triggering a build.The certificate of the test assembly need to be installed on the build agent machine. This is a one time operation and can be done by one of the following ways :


From an admin cmd prompt on the build agent box, execute “Certutil -importpfx *.pfx”  where the pfx file could be found in the project folder of your test project. Note the password for the default pfx file is blank. Also, you can use the same pfx file for all your test projects (instead of having multiple files). This can be done by setting the <PackageCertificateKeyFile**.pfx</PackageCertificateKeyFile> node in the csproj file.

Else, install the certificate as part of post build event using the following steps :

  a. Update the csproj file so that post build event gets executed only in the case of Teambuild and not local build.

  b. In post build event specify a script to execute “Certutil -addstore root *.cer” where the *.cer file could be found in a sub folder in the build drop. Create a wrapper script  which would get the full path of the .cer file present in the build drop and pass it to the Certutil command.

Trigger a build and analyze the test results:

Now queue a build and analyze the results of your metro style tests.


Frequent errors and resolutions:

  1. Could not start tailored test run because Unit tests for Windows Metro style app cannot be run from a service or non interactive process. Please run unit tests from an interactive process.
    1. This error occurs if the build agent is running as service.
  2. Access is denied. (Exception from HRESULT: 0x80070005
    1. This error occurs if the logged in user in the build agent machine is different from the build agent interactive process account . Make sure that the logged in account and the build agent account are same for Metro style apps to be run properly.
  3. Installation of package 'C:\Builds\1\proj1\CalcBuild\Binaries\CalculatorTest\AppPackages\CalculatorTest_1.0.0.0_AnyCPU_Debug_Test\CalculatorTest_1.0.0.0_AnyCPU_Debug.appx' failed with Error: (0x80073CF1) Package was not found. (Exception from HRESULT: 0x80073CF1).
    1. This error occurs if the certificates are not installed on the build agent box. Make sure that you install the certificate either manually (one time) or using post build scripts. Also make sure that the certificates of newly added test projects are also installed. [ or have all test projects point to same pfx file which is installed one time]
  4. Code coverage collection with metro style tests

                       Code coverage doesn’t work with Metro style tests ( either in VS or in Team build or via the command line runner).

    Skip to main content