Test run remains in “Waiting for test controller” state for ever


In this article, I wanted to share what you should do if you have queued an automated test run on a test controller and its remain in "waiting for test controller” state for quite some time (as shown below). 

WaitingForTestRun

Before we go to the trouble shooting steps, let us spend sometime understanding what happens when you want to queue a test run.

You queue a test run specifying the test environment on which the test run should get executed . For example: – For the test run shown in the above picture, I have queued it against a test environment named “aseemb”.  This queuing of test run essentially means that the run gets stored on the team foundation server and is ready for execution on its test environment. Note that here only the run is ready for execution and not the test environment. The test environment on which you have queued the run might be ready or might not be ready.

The “test environment” is managed by test controller and the test controller is the one who is responsible for managing the execution of the queued runs. Test controller is supposed to pick up the queued runs from the team foundation server and execute them on the environment for which the test run is queued. The logic to pick up the test run is intelligent and it does not pick up the runs which are queued against the environments that are not ready right now and picks up only those runs which are queued against ready environments. The runs which are not picked up by the controller remain in “waiting for test controller” state and the runs which get picked up move to “In Progress” state.

Now let us go back and find out what you should do if you observe that the test run has been in “waiting for test controller” state for quite some time.

1.  Ensure that the test controller is running and is able to communicate with team foundation server (meaning there is no network issue, permission issue etc).

2. (Update on 09/11/2011) Ensure that all the agents in the environment on which you have queued the run are in ready state.

Allagents

3. (Updated on 25/06/2010) Ensure that the test controller account has “View Test Run” permission on the project in which the test run is queued. This is a very rare miss and typically the problem is one of first two options. But in case you manually change the controller account (services.msc) or you have custom groups & have manually edited the permissions, then there is a chance that this permission is not there for the test controller account. To check whether the appropriate permission is there or not, you can run the utility (source code is here) with following parameters.

CheckProjectPermission /collection:http://mytfs:8080/tfs/DefaultCollection /teamProject:P1

                                  /user:mydomain\myaccount /projectPermissionName:ViewTestResults

Comments (14)

  1. I have noticed that even with just one test it can take a few minutes to clear the "Waiting" status. Be patient. However, it would be nice if the performance in this area could be improved.

  2. Jaredow says:

    I noticed that the test agent machine's status is "Offline" when you log into it while in Test Manager its says its waiting. "Resetting" the Test Agent speeds up the process of it executing tests

  3. Jaredow, when the agent machine's status is offline, then the environment is not ready to run the tests. And when environment is not ready to run tests, test run remains in waiting for test controller state. And when you reset the agent state to online, then environment becomes ready and the test run is picked up for execution.

  4. SaintD, We have done couple of performance improvements in this area and they are available in a QFE with details mentioned on the following link.

    blogs.msdn.com/…/new-rollup-qfe-for-various-issues-in-mtm.aspx

    Please try it this QFE on the test controller machine and let me know how it works for you.

  5. View Test Results permission says:

    Hi,

    I checked and saw that my test controllers account does not have View Test Result permission.

    But i couldn't find how can i give that permission?

    There is notthing like View Test Results in TFS Administration Console's Administer Security section.

  6. I am out of office these days, so I cannot give you the exact name of the permission. But there is a permission at the project level with name like "View test run" (or  "Create test run") which maps to ViewTestResult permission.

  7. Hi,

    I have the same problem, the test run remains in waiting for test controller state forever.

    I can see the record in the TestRun table which enters with state 1 and then updated to 5. once it is in state 5 it remains like this forever.

    The agents are ready and there are no errors in the even viewer.

    Any idea what might be the reason?

  8. Make sure that you have installed Test Agent to your Virtual Machine.

  9. Richard says:

    My tests normally start within a few seconds or minute at the most. Occasionally, the test run can take over 2 hours to start.  What would cause that?

  10. Thanks Richard for reaching out. Does any of the problems mentioned above applicable when it takes 2 hours to start?

    Regards

    Aseem Bansal

  11. Kevin Alder says:

    I am seeing the same thing where Tests take forever upwards of 15 minutes to 30 minutes to start with TFS 2013 when I have a TFS 2012 Test Controller and Agent. Ideas?

  12. Ajay Arora [MSFT] says:

    Possible reasons why your testrun could take more time :

    1. Deployment Stage : TestController needs to copy build drop from the build drop location to the TestAgents. Make sure that your deployment bundle (build drop, test binaries, test helper binaries, deployment items etc) is not big, and your network is not leading to this delay. (for ex. TestController and BuildDrop location are not in the same network. Or Testagent and TestController are not in the same network)
    2. Agent Initialization : If you have enabled lots of data collectors in your test settings.

    Additionally, you will be able to spot the time delay in your test controller logs. Please enable verbosity level logging on both TestController and TestAgent machines using the below links. If you need any help, you can mail me at ajay.arora@microsoft.com or reply here.

    blogs.msdn.com/…/how-to-enable-test-controller-logs.aspx

    blogs.msdn.com/…/how-to-enable-test-agent-logs.aspx

  13. Richard says:

    The 2 hour delay is not reproducing at the moment. But we are seeing 7 minute delay. Our deployment happens before the long wait for a test controller. Done as part of a build test deploy workflow. We don't have any data collectors enabled. I enabled logging on the controller and test machine.  Not sure what to look for.

  14. Maria says:

    Another thing to check is that you have chosen the right Test Settings and Test Environment for this automated test run. This is how I resolved my problem. Before, I was using a Test Environment that wasn't ready, by mistake.