Driving Automation of Test Cases

As I’ve mentioned in previous posts, our team has been working our way up the testability pyramid. We started with unit tests, and now have about 670 C++/CLI unit tests that are testing our C++ code. However, we didn’t have any automated acceptance or UI tests. Our tester was going nuts because he was always behind the curve, and was doing entirely manual testing.

Starting about two sprints ago (our sprints are two weeks), we finally had both the resources and infrastructure to be able to start automating acceptance tests. I’ll describe our infrastructure in a separate blog post. But once we start write tests, we wanted to know how were were doing. To answer that question, I wrote a Reporting Services report:

You can download this report from here: Test Automation Status.rdl.

The report above is real data from one of our projects. Green is good—it shows how many test cases we’ve automated.

This report pulls data from the Test Case work item’s Automation Status field. This field can be Not Automated (the default for new test cases), Planned, or Automated. You can switch to automated by clicking on the Associated Automation tab of the Test Case and selecting a test method that is in the currently open solution. Alternatively, if you have the Test View tool window open in Visual Studio, you can right click a test, click Associate Test to Test Case and then choose a test case.

Since we plan to automate almost all of these test cases, we’re using the Planned automation state to represent test cases where we’re writing an automated test. In other words, in our case, Planned represents work in flight.

Hope you find this report as useful as we do in both driving test case automation, and knowing where you’re at.