Running VSTS unit tests on WM5 devices

Porting Code and Unit Tests to the device:

After much work, we ported the source code and unit tests from the desktop versions
of CompositeUI, CompositeUI.Winforms, and ObjectBuilder projects to
corresponding Windows Mobile 5.0 Pocket PC projects. We even got the unit
tests to pass when run on the VSTS unit tester. We're done right? Wrong.

In order
to get a real sense of completion, we need to run our unit tests on the device,
as opposed to the VSTS test runner that runs on the desktop. After some
searching, we did not find a suitable tool, so we wrote our own: CFUnitTester.

After
running the Mobile CAB and ObjectBuilder unit tests on a device via
CFUnitTester, we found remarkable differences in the way our code ran on the
desktop versus on a device. One difference being that on the desktop, there is
an app domain versus none on the device. After fixing our code and updating our
tests, we were finally able to get all unit tests to pass on both VSTS test
runner and CFUnitTester.

Looking
back at this experience, I could not imagine how long it would have taken to
port CAB and ObjectBuilder if it weren't for the large number of unit tests.
Thank you CAB and ObjectBuilder teams for your test driven development. You
saved us months of work!!!

Adapting
VSTS Test Classes to Run on CFUnitTester:

1. Assuming you have a
   VSTS test project named <MyAssembly>.Tests, create a WM5 Pocket PC
   Class Library project named <MyAssembly>.Tests.CF. Move the project
   file into the same folder as your VSTS test project. This is so that both
   projects can share the same files & test classes.

2. In Visual Studio
   Solution Explorer, select the device test project you just created and
   click the "Show All Files" button. You should now see the files
   in the VSTS test project. Select the test classes (typically named
   <MyClass>Fixture.cs) and supporting files and select "Include
   in Project" from the Right-Click menu.

3. Add a reference to
   the Microsoft.Practices.Mobile.TestTools.UnitTesting assembly which is one
   of the three assemblies in the CFUnitTester.

4. Add reference(s) to
   the projects/assemblies that your tests depend upon, matching the
   references in the VSTS test project with the exception of
   Microsoft.VisualStudio.TestTools.UnitTesting which is for VSTS desktop
   testing.

5. You will now need to modify every test class in your
   test assembly to dynamically use the CFUnitTester if the test is
   running on the PocketPC. Wrap the using
   statement that references the VSTS UnitTesting
   assembly.

   #if PocketPC

   using
   Microsoft.Practices.Mobile.TestTools.UnitTesting;

   #else
   using
   Microsoft.VisualStudio.TestTools.UnitTesting;
   #endif

6. You should now be able to successfully compile.

7. Now, add the GuiTestRunner and TestRunner projects from
   CFUnitTester to your solution.

8. Deploy the GuiTestRunner project. This should deploy the
   GuiTestRunner and supporting assemblies to the device folder: Program
   Files\GuiTestRunner

9. Edit the properties for your new
   device test project... On the "Devices" tab,  change the
   Output file folder to: %CSIDL_PROGRAM_FILES%\GuiTestRunner

10. Deploy
    your new device test project.

11. On
    the device, open File Explorer and navigate to the Program Files\GuiTestRunner
    folder and tap on the GuiTestRunner executable. This should reflect over all
    assemblies in the same folder looking for test classes and test methods,
    TestClassAttribute and TestMethodAttribute respectively.

12. The
    GuiTestRunner UI should display your new device test project and associated
    test classes & methods. Select a test project by tapping the appropriate
    checkbox and tap the Run Selected button.

13. Examine
    the results in the Results tab. The GuiTestRunner also dumps the results to
    Program Files\GuiTestRunner\log.txt on the device.

If you need to debug your tests, simply set the GuiTestRunner as the startup project, and F5.
Enjoy.