Update TestSettings utility


In dev10/dev11, test settings in MTM does not expose all the configuration options which are present in Visual Studio. These configuration options include bucket size, deployment timeout among few others.

I have created a small utility named UpdateTestSettings which you can use to update these parameters. To start with, it provides options to update/view the bucket size, update/view the deployment timeout and going forward I will add more options to this list based on your requirements. In case you want to extend it your selves, you can use the 2010 source code from here or 2012 source code from here or 2013 source code from here.

Here are the steps which you need to do to use this utility: –

– Download/extract this utility from here for 2010 version, from here for 2012 version and from here for 2013 version. Ensure that the machine has appropriate version of Visual Studio or Microsoft Test Manager installed.

– Open a command prompt, change the directory to the one where you have extracted this utility and run it using the following syntax.

UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>]

Examples:

UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000

UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProjec:myProject /settingsname:My2_0_App

Comments (26)

  1. Hi Need Help on MTM says:

    Hi I need to know about MTM , How it works for performance testing. i have installed it..noe i need to use it for Load and Performance testing need help urgently…!!

    Digvijay Singh

  2. Digvijay,

    MTM is not meant for Load / performance testing. You need VS Ultimate for it. You get can VS Ultimate trial version from here(add link) in case you don’t have it.

    Please refer to below walkthroughs for details on how to do Load / Performance Testing using VS Ultimate:

    Web test Walkthrough:

    msdn.microsoft.com/…/aa337591(v=VS.100).aspx

    Load Test Walkthrough:

    msdn.microsoft.com/…/ms182594(v=VS.100).aspx

    And you can post your questions around Load / Performance testing on this forum.

    social.msdn.microsoft.com/…/threads

    Regards

    Aseem Bansal (on behalf of Rubel)

  3. Nikhil K Narendran says:

    After the settings, do we need to restart the Microsoft test manager?

  4. Nikhil, The settings page in MTM should be refreshed to get the updated settings from the server. I think that should be enough and restart of MTM should not be required.

    Regards

    Aseem Bansal

  5. Nikhil K NArendran says:

    Hi Aseem,

    Thanks for your info.

    I used update time settings and i have given 100000.

    I am using one batch file to install application before my test case execution, some times its taking more time to install, so getting 'The setup batch file on agent machine <machinename> exceeded the execution time period." But it is taking maximum 7 minutes.

    Even i have given 100000 also we are getting exceeded time period in agent.

    Could you please help me on this.

    Thanks

    Nikhil

  6. To update the script timeout, you should modify the code of this utility as discussed in the below thread.

    social.msdn.microsoft.com/…/5a6f3af4-9e28-42d9-8830-f800969d8b28

    Regards

    Aseem Bansal

  7. Nikhil K Narendran says:

    Thanks a lot Aseem. It is working now.

  8. Asher Barak says:

    Thanks

  9. Meenakshi says:

    Can you please tell me what is deployment time out? what is the use of it?

  10. The testcontroller deploys the test binaries and their dependencies from the build drop location to the test agent machines so that test agent can run them. The timeout for that activity is called deployment timeout.

  11. How it will work with Remote Agents and Controler says:

    When I execute load test .. parallel execution (2 users use the same Test Machine),but they executed two diffrent scenarios with 1 contrroler and agents. Only one Instance has Active another instance has shown on other user machine pending. how could i run both parrlael on shared test machine

  12. Ryan says:

    Aseem:  Since your app works on versions < 2013, I need to try to create an updated version.  Can you point me to anything that describes where the Bucket Size property is exposed in the TFS api?  I appreciate your help.

  13. Venkata Sadineni says:

    Question:  

    #1. I know in TFS Build on VS, we can updated the parallelTestCount attribute which allows to run multiple tests, but I do NOT find this attribute exposed in the TestRunConfiguration class. How can we configure so that multiple tests run on a given agent.

    #2.

    In MTM test environment with multiple test agents, can the tests run in Pararallel on a test agent.

    Eg: Test Env1: 5 test agents, 1500 test cases. I know that each agent will be running 1 test at a time. So at any given time, we have 5 tests running.

    Instead can I run 10 tests on test agent, so a total of 50 will be running at any given time??

    I know in TFS Build on VS, we can updated the parallelTestCount attribute which allows to run multiple tests, but I do NOT find this attribute exposed in the TestRunConfiguration class.

  14. Geetika says:

    I am receiving error: The setup batch file on agent machine 'XX' exceeded the execution timeout period. Although the difference between test start time and test complete time is 8 min.

    In test setting, the time-out period is mentioned as 30min (Mark an individual test as failed if its execution  time exceeds : 30min.

    This was running fine last few days and suddenly I see this error. Wondering why am I seeing this error. Could you please help?

    Thanks Geetika

  15. Geetika says:

    Hi Aseem,

    The right side text in your article is being cut. Could you please mention again the command you are using for the utility. And what is bucket size?

    Thanks,

    Geetika

  16. Geetika Bangard says:

    and I have VS 2013. Will it work with this one? Or should I install 2012?

  17. Here you go: –

    UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>]

    Examples:

    UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000

    Secondly it should work with VS 2013 as well but you might have to recompile the sources with 2013 which should be simple.

    Regards

    Aseem Bansal

  18. Srini says:

    Hi Asim,

    This tool doesnt seem work with TFS2013, we just upgraded to TFS2013 and i am trying to set the bucket size to 1, we have 100 clients lab environment but test controller distributes tests only to last 7 clients instead of all 100. ( it was working well with 2010). lets say we have 500 test cases, it distributes one test to each of the client. once client finishes test it takes another test from test controller.

    can you help to find out whats going on?

    Thanks

    Srini

  19. Srini there are no changes in the distribution logic. If you can share your testsettings file it would help.

    Beyond that you can check the status of your agents prior to the run and see if that shows any hints of any issue.

    Another thing to check is if theses last 7 clients have something in common – in comparison to the others. That should also help us correlate back to what we know.

    if none of this is providing you with any help. I would recommend enabling the test controller traces and then starting your run. The traces should have data if any agent is being skipped.

  20. Srini says:

    Hi Allen,

    Now its working after i put the app key settings MinimumTestPerAgent as 1 in test controller.

    Now we are having different issue, after setting up the environment when i connect to machines thru lab viewer after certain time the machines gets locked, we have to enter password to open it again. we are using windows 2012R2 as hyper-v servers and clients machines are virtual machines with win7 on it.

    anyone experienced this issue?

    Thanks

    Srini

  21. ND Developer says:

    Is there overhead or poor performance impact associated with setting the Bucket Size to a very small number such as 1 or 2?

  22. chamindac says:

    I have added ScriptTimeout to the tool and made it available here. you can use as follows

    UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>] [/scriptTimeout:<value>]

    Examples:

    UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000

    UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000 /scriptTimeout:900000

    UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProjec:myProject /settingsname:My2_0_App /scriptTimeout:900000

  23. chamindac says:

    link for downloading it here.. gallery.technet.microsoft.com/Update-Script-Timeout-in-c40af3ee

    UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>] [/scriptTimeout:<value>]

  24. Woops says:

    How do I delete comments? that link is actually fine.

  25. Salami Army says:

    Is there a way to do this without TFS? My test rig is made up of VS2013.5 test controller and agents, no TFS, but I'm still getting script timeouts because my application takes exactly 7 minutes to start. How do I increase the script timeouts in my .testsettings files?

Skip to main content