What does deployment do when running a test?

  • Article

A lot goes on in deployment. Michael Koltachev, developer of the feature, recently sent an email explaining it to one of our testers:

· In Orcas we introduced a way to run tests with deployment disabled. In this case, we execute the tests from where they are, such as bin\debug directory. There is persisted public property exposed in Run Config UI: IsDeploymentEnabled. When deployment is disabled, certain scenarios are disabled as well, for instance “rerun original tests” is disabled.

· For local runs, only 1 deployment directory is used for all components, TMI, Controller, Agent.

· For Remote runs, TMI uses local directory, then Controller uses different local directory on the Controller machine, then Agent uses different deployment directory on the Agent machine. It does not matter, if these all are the same machine.

· TMI deployment:

o what we deploy and the order:

§  Tests storages (the files that tests are stored in)

· Executable files (e.g. unit tests and coded web tests)

· PDB files

· XML files (.webtest, .loadtest, etc.)

·

.config files

§ Check whether deployment is from a trusted location (we require full trust on the “deploy from” files for test storages).

§ Code coverage items (including those for in-place coverage)

§ Run Config deployment items

§ Deploy Dependencies of test storages

· We check all dependencies recursively from assembly manifest file.

· There is a way to disable dependency check for certains assemblies: use persisted regex property RunConfig.IgnoredDependentAssemblies.

§ Per test deployment items

§ If deployment of satellite assemblies is enabled (disabled by default for perf) in Run config, find & deploy them.

§ For all deployment items we deploy PDB files (introduced in Orcas).

§ Generate warnings for missing dependencies:

· If there is a file with the same name and timestamp deployed from different directory to the same output directory,, we don’t generate a warning.

· We generate warnings for all missing dependencies.

o Where we look for files:

§ It the item has explicitly specified path, we use it

§ Use RunConfig.SearchPaths, which is Solution Directory for IDE and “.” for command line. Also I don’t remember what happens in the big build scenario but as far as I remember, it specifies some explicit search paths.

§ For each test storage processed, add directory of where it is located to the search paths

§ Add the directory of where .testrunconfig file used is located to the search paths.

o Differences IDE from command line

§ When run form IDE, all references recursively of current test project are added as deployment items

o Other notes

§ NC introduced ITestElementDeploymentRuleProvider (implemented by a TIP on the same instance that implements iTestElement)

· ImpliedDeploymentItems (e.g. a web test can specify additional items)

· ImpliedDeploymentAssemblies (similar to above but the items that are checked for dependencies)

· RequiresFullDeployment.

· Reverse deployment:

o In the end of test run some files need to be deployed back from Agent to the client.

§ Coverage result files

§ Any other files put by a test or Agent/Controller to the In directory.

§ For local runs, all files are placed to the same dir, “In”

§ For remote runs, we create subdirs under each “In” directory for each agent, such as: “In\Agent1”, “In\Agent2”. The idea is that otherwise the files may collide for different agents. Then we merge files from all agents.

· Deployment between TMI and Controller, and between Controller and Agent

o Deployment is done before test run is started.

o We don’t’ use File.Copy, we use file streams over .Net Remoting channels.

Ed.