Ordering Method Execution of a Coded UI Test

There’s often no discernable, consistent pattern that dictates the execution order of automated tests (Coded UI in this example, but the same applies to Unit Tests).  Some argue that it may be a good thing that there isn’t an inherent pattern for the execution of CodedUI tests (and unit tests, for that matter), as a seemingly more random pattern can uncover test dependencies which reduce the overall effective coverage of tests.  And I agree with that to an extent, but there are always cases in which control over execution order is needed.

An easy way to accomplish test order is to use an Ordered Test. This will provide you explicit control over the execution order of your tests.

For this example, I have a Coded UI Test class called CodedUITest1 (for more on Coded UI Tests, see the Anatomy of a Coded UI Test).  In it, I have two CodedUI Test methods:

  • CodedUITestRunFirst()
  • CodedUITestRunSecond()

I want to order them such that they execute like:

  • CodedUITestRunSecond()
  • CodedUITestRunFirst()

1. Add a new Ordered Test. Go to Test->New Test, and select Ordered Test.

 New test window

2. The ordered test will open. I can move the available tests from the left list into my ordered test list on the right. I can then move the tests up/down to create the desired order.

 Ordered Test dialog

It’s not shown in this screenshot, but there is a checkbox to allow the ordered test to continue upon a failure.

3. Save the ordered test. I can now see the ordered test in my Test View window.

Test View window showing new ordered test

4. When ready, I select to run my ordered test. It will appear in the Test Results window as a single test.

Test Results window showing ordered test

When finished, I can double-click on the test result to see that both tests did actually run, their individual results, and their order.

Detailed results of ordered test

It’s a surprisingly easy yet elegant solution.  I can put pretty much any automated test into an ordered test (except for load tests).  If you have a lot of tests, coupling the use of ordered tests with other test lists can really help visually organize your test project.

Comments (4)

  1. Steve, I appreciated your blog about "Ordering Method Execution of a Coded UI Test".  But I need to do the same thing from a build definition and a remote execution.  Could you please tell me how to "Order" the sequence of my test methods in a build definition file?  Thank you.  Vance Bradford vancebradford@pobox.com

  2. deepak says:

    As per my knowledge regarding ordertest  is , i can create the test order list and past it in argument to MSTEST.exe.

    I want to control my orderlilst before executing the test. Because i have requirement where i m going to change the list <some times i want to avoid certain test cases> . Hence I wanna control where I can handle any XML or xls file in which i will provide the order list and then i need option through which I can modify my order test just before execution.

    I have user of my automation who don;t know much about order test or you can say i don;t want them to open VSTS on their test machine. They will only open XML or xls to control the number and order of execution .

    Can some one give me any hint.

  3. deepal says:

    I have use order test and yes it working.

    Now my question is bit extended. Suppose u have one module have 5 test cases which have their own class initialize and class clean up. now other module have same 10 test case with class initialize and clean up .

    Issue is when i club all 5 +10 test cases, in that scenario after my 5th test when my 6th test or order test (1st test case of 2nd module ) execute with initialize of 2nd module  but clean up of 1st module not executed. Which cause an issue . because it impact on my 2nd modules.

    In short in order test if some test cases have one class initialize and clean up while rest have their own different initialize n clean up then how it work ?

  4. Mohammad Saad says:

    @deepal, in the test initialize method of your second module, call the test initialize method of first module.