Running Windows Phone Unit Tests via MSBuild

  • Article

I’m a big fan of TDD and continuous integration, so when I first started development on Windows Phone 7, I quickly found Jeff Wilcox’s Silverlight Unit Test Framework. This unit testing framework provides a nice way to write MSTest style unit tests and have them run on a device/emulator. The piece that was missing for me was the ability to capture the results of the unit test run and include these results with an automated run of MSBuild. If any unit tests fail, the MSBuild run should fail.

I’ve seen several blog posts on this topic but none that gave me an end to end solution. Armed with the latest Mango bits and the CoreCon 10 WP7 API, this task was easier than I thought. Here’s how it works…

  1. Capture the unit test run with custom LogProvider that writes the test run messages to a file in isolated storage.
  2. Custom MSBuild task that deploys test xap file to emulator, installs the application, runs the application, waits for a configured number of seconds and harvests the results of item #1 above.

Let’s start with the custom LogProvider:

 public class FileLogProvider : LogProvider
{
    public const string TESTRESULTFILENAME = @"TestResults\testresults.txt";
    protected override void ProcessRemainder(LogMessage message)
    {
        base.ProcessRemainder(message);
        AppendToFile(message);
    }

    public override void Process(LogMessage logMessage)
    {
        AppendToFile(logMessage);
    }

    private void AppendToFile(LogMessage logMessage)
    {
        UTF8Encoding encoding = new UTF8Encoding();
        var carriageReturnBytes = encoding.GetBytes(new[] { '\r', '\n' });

        using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
        {
            if (!store.DirectoryExists("TestResults"))
            {
                store.CreateDirectory("TestResults");
            }
            using (IsolatedStorageFileStream isoStream =
                store.OpenFile(TESTRESULTFILENAME, FileMode.Append))
            {
                var byteArray = encoding.GetBytes(logMessage.Message);
                isoStream.Write(byteArray, 0, byteArray.Length);
                isoStream.Write(carriageReturnBytes, 0, carriageReturnBytes.Length);
            }
        }
    }
}

This class basically receives log messages and appends them to the TestResults\testresults.txt file in isolated storage. Pretty straightforward. You’ll need to reference Jeff Wilcox’s Silverlight Unit Test Framework in order to resolve LogProvider.

 

You use FileLogProvider in your test project. Here is an example MainPage.xaml.cs

 

 public partial class MainPage : PhoneApplicationPage
{
    // Constructor
    public MainPage()
    {
        this.InitializeComponent();
        LogProvider fileLogProvider = new FileLogProvider();
        var settings = UnitTestSystem.CreateDefaultSettings();
        settings.LogProviders.Add(fileLogProvider);

        Content = UnitTestSystem.CreateTestPage(settings);
        BackKeyPress += (x, xe) => xe.Cancel = (Content as IMobileTestPage).NavigateBack();
    }
}

Great! Now that we’ve captured the test run results on the phone, how do we get it to the server? I’ve heard people use web services but I found using the CoreCon API worked fine.

My custom MSBuild task does the following:

  1. Connects to the emulator
  2. Installs/Reinstalls the application
  3. Launches the application
  4. Waits a specified number of seconds for the application to finish running tests.
  5. Pulls the TestResults\testresults.txt file from the application’s isolated storage and copies it locally.
  6. Examines the contents of the test results for failing unit test.

Here is the source for my custom MSBuild task:

 namespace MSBuildTasks
{
    public class RunWP7UnitTestsInEmulator : Task
    {
        private int defaultNumberOfMiliSecondsToProcess = 50000;

        [Required]
        public string PathToUnitTestXapFile { get; set; }

        [Required]
        public string ProductGuid { get; set; }

        public int NumberOfMilliSecondsToProcess { get; set; }

        public override bool Execute()
        {
            if (NumberOfMilliSecondsToProcess == 0)
            {
                Log.LogMessage(string.Format("Using default processing time: {0}", defaultNumberOfMiliSecondsToProcess));
                NumberOfMilliSecondsToProcess = defaultNumberOfMiliSecondsToProcess;
            }

            Log.LogMessage(string.Format("NumberOfMilliSecondsToProcess: {0}", NumberOfMilliSecondsToProcess));

            Log.LogMessage("Running tests in XAP: " + PathToUnitTestXapFile);
            var productGuid = new Guid(ProductGuid);

            var emulator = GetWP7Emulator();
            Log.LogMessage("Connecting to Emulator.");
            emulator.Connect();
            Log.LogMessage("After call to Connect().");

            //Deploy application
            if (emulator.IsApplicationInstalled(productGuid))
            {
                emulator.GetApplication(productGuid).Uninstall();
            }

            emulator.InstallApplication(productGuid, productGuid, "NormalApp", null, PathToUnitTestXapFile);

            //Run application
            var application = emulator.GetApplication(productGuid);
            application.Launch();


             Thread.Sleep(NumberOfMilliSecondsToProcess);

            //Get Results from Isolated Store on device
            var isostorefile = application.GetIsolatedStore();
            var localTestResultFileName = "EmulatorTestResults-" + productGuid + ".txt";
            isostorefile.ReceiveFile(@"\TestResults\testresults.txt", localTestResultFileName, true);

            var testResults = File.ReadAllText(localTestResultFileName);

            if (testResults.Contains("Exception:"))
            {
                Log.LogError("Failing test found. Look for exception in: " + localTestResultFileName);
                return false;
            }

            Log.LogMessage("All tests passed.");

            emulator.Disconnect();

            return true;
        }

        private Device GetWP7Emulator()
        {
            var manager = new DatastoreManager(CultureInfo.CurrentUICulture.LCID);
            var wp7Platform = manager.GetPlatforms().Single(platform => platform.Name == "Windows Phone 7");
            return wp7Platform.GetDevices().Single(device => device.Name == "Windows Phone Emulator");
        }
    }
}

 

You’ll need to reference CoreCon (Microsoft.SmartDevice.Connectivity.dll). On my 64 bit machine, I found the dll here:

C:\Program Files (x86)\Common Files\microsoft shared\Phone Tools\CoreCon\10.0\Bin\Microsoft.Smartdevice.Connectivity.dll

Finally, I just call my build task in an msbuild proj file:

 

 <Project DefaultTargets="Test" xmlns="https://schemas.microsoft.com/developer/msbuild/2003" >
    <UsingTask TaskName="MSBuildTasks.RunWP7UnitTestsInEmulator" 
        AssemblyFile="MSBuildTasks.dll"/>

    <PropertyGroup>
        <SourceFilesPath>$(MSBuildProjectDirectory)\..\Source\</SourceFilesPath>

    </PropertyGroup>

    <ItemGroup>
        <SolutionItemsToBuild Include="$(SourceFilesPath)\**\*.sln" />
    </ItemGroup>

    <Target Name="Build">
        <MSBuild Projects="%(SolutionItemsToBuild.Identity)" Targets="Build"/>
    </Target>

    <Target Name="Test" DependsOnTargets="Build">
        <RunWP7UnitTestsInEmulator NumberOfMilliSecondsToProcess="50000" 
            PathToUnitTestXapFile="$(SourceFilesPath)\MyWP7Project.Tests\Bin\Debug\MyWP7Project.Tests.xap" 
            ProductGuid="MyWP7ProjectProductGuid" />
    </Target>
</Project>

You’ll need to replace the PathToUnitTexstXapFile and ProductGuid, but this proj file should build all solutions in the “Source” folder and then test the Windows Phone unit tests specified in the Test target.

I’ve had this integrated into my continuous integration server and it works great. I need to run the CI server in interactive mode because launching the emulator on my server box always pops a dialog complaining about the graphics processing unit configuration. I just keep an emulator instance open to avoid this popup.

image