Running Tests cross domain with TFS and Visual Studio 2010

  • Article

In part 1 I promised to continue the configuration to allow running tests from TFS with Microsoft Test Center. In order to run tests cross domain, you need to have completed Part 1.

The best practices for work and production networks require some seperation between the networks. This is great for security, but can make using the tools in Visual Studio and TFS 2010 a problem. This post will show you how to use Lab Management and "no more no repro" bug data to get your job done easier and faster.

Pre-Requisites:

  • Your team using Visual Studio 2010 and TFS 2010

  • You build in TFS 2010

  • Your TFS server is accessible on port 80 from an edge machine in your test network. (Check the IPSEC policy)

  • You have setup and tested using a test controller and test agents according to my previous post. Important!

  • Your test controller is on an edge machine joined to your test domain, but visible to your work domain.

  • Your test controller is runnng on a port that is open from your corp network to your test newtwork. (80 and 443 for example, might be open by default.)

  • You have admin access to the TFS server.

  • You have a TFS project setup and have a product build and some tests that build.

  • You are an administrator of the TFS project you want to connect to.

Assumptions:

  • All Machines are Windows 2008 Server r2
  • You have admin access to the machines.

Connecting TFS to the test controller cross domain

Warning: If you connect a test controller to TFS you may not be able to use it with Visual Studio in the future when you disconnect it. Disconnecting it will destroy any envronments you set up anyway. Be sure to have a dedicated controller for Visual Studio if you want to run tests directly from there. Connect Bug ID:585640

These instructions assume you don't have two way trust between the domains. If this is the case, you can still use lab management, but you won't be able to use SCVVM at all. You will be limited to "physical" environments. These can contain Virtual Machines and the setup isn't hard. You don't get the ability to roll test environments back, or create copies. However, it's well worth the benifits of rich debugging of automation failures.

  • On the TFS server.
    • Setup a machine account that matches the account name and password used for your test controller. .\TestControl was my suggestion in my last post.
    • Add that account to the admin group on the machine.
    • Launch Visual Studio 2010 and open the team explorer window.
    • Right Click your TFS root node (you may have to run VSTS as admin) and select "Team Project Collection Settings -> Security"
      • Add the .\testControl account to "Project Collection Service Accounts" and "Project Collection Test Service Accounts".
  • On the test controller machine
    • Open a command prompt in the directory where TestControllerConfigUI.exe is installed. "..\Program Files\Microsoft Visual Studio 10.0\Common7\IDE" for example.
    • type the following command, replacing machinename with the name of the local machine.
      •  runas /user:%COMPUTERNAME%\testControl TestControllerConfigUI.exe
    • Enter the account info using the format .\testControl.
    • Enter the FQDN to the TFS server. Use the Browse button to help construct the URL. E.G. TFS..corp.mycompany.com:8080/tfs/defaultcollection
    • (Optional) configure a load testing database. You can always add this later.
    • Select "Apply Settings" and wait for tasks to complete.
    • Determine the FQDN the machine has for itself.
  • Back on the TFS server.

    • Ensure the FQDN for the test controller resolves to an address the TFS server can connect to.

    • If the FQDN is in another domain, add an entry to the TFS server HOSTS file for your test controller.

    • Ensure you can connect to the controller port from the TFS server. You may need to adjust the firewall settings on both machines.

Creating Test Environments

Now you may start connecting test machines to the controller and adding them to environments in Lab Management. Assuming you have some test agents connected to the controller, this is how to set up an environment. (Automated deployment is out of scope for this post, the build controller needed can be put on the same edge machine using the same account.)

  • Open Microsoft Test Manager 2010 and make sure it's connected to your TFS server and project.
  • Click the arrow next to "Testing Center" and select "Lab Center"
  • Make sure the "Controllers" tab is selected. Verify that your edge server controller is online. Click the controller and verify your test machines are online.
  • Make sure the "Lab" tab is selected.
  • Click "New -> New Physical Environment"
  • Name and describe the environment.
  • Select the correct test controller.
  • Click "Next"
  • Select the machines you want to add to the environment and select "Add to environment"
  • Select the approriate role for each server.
  • Click "Next"
  • Add any appropriate tags to the machines. You can always come back later and add them.
  • Click "Next.
  • Click "Finish"

Adding a Test Enviroment to a Test Plan

If you are planning on having several of environements, you may want to organize your test plans one per environment.

  • Go to the "Testing Center" mode.
  • Select the "Organize Tab"
  • Select a test plan, or create a new one to go with your environment.
  • Set the "Test Enviroment" under "Automated Runs" to be the enviroment you just set up.
  • Pull down "Test Settings" (right above Test Environment) and select "new".
  • Name the settings. You can match the name to the test plan.
  • Make sure "Automated" is selected.
  • Click "Next", Skip the roles page, click "Next"
  • Select each role and check off the diagnostic infomation you want to collect. (Video recording require Windows Media Encoder 9 to be installed on the agent.)
  • Click "Finish".
  • Go to the "Build In Use" modify and select a build with your automated tests and click "Assign to plan."
  • Click the "Home" button and ensure the test plan you just configured is selected.
  • Go to the "Plan' tab.

Running the Automation

  • Make sure you have added some automated test cases. (Test cases must be checked in and built on TFS).
  • Go to "Test" tab.
  • Select a suite or an automated test and select "Run".

Your test will run and you can analyze the test results and log bugs with the rich debugging information. Bugs logged this way can be debugged in your Corperate environment without needing a connection to the test machines.