Assembly Resolution for Unit Tests

A common problem in writing test automation against an API is that you need to be able to load the assemblies containing the code under test (CUT) into the test process.  There are several options which have been employed in the past to ensure that it is possible to load these assemblies including copying the CUT assemblies next to the test assemblies, copying the test assemblies next to where the CUT assemblies are deployed before running the tests, and setting up custom assembly resolvers.  Each of these has their drawbacks when you start trying to execute your tests in different environments.  For example copying the CUT assemblies next to the tests may work great on your development machine, but when you want to also run these tests in a lab against the installed product a better solution is needed.  In VS 2010 we have incorporated an assembly resolution solution directly into the unit test framework which will allow you to easily configure where the dependencies of your tests get loaded from.

The assembly resolution settings can be configured by opening up Test Settings and navigating to the “Unit Test” section.  Below is a screenshot of the settings for reference as we go through what each of them are for:

These assembly resolution settings will be used for all tests that are executed in the run and will override any assembly resolution settings which were put into the test projects App.Config file (more later in this post about the assembly resolution settings that can be put into the App.Config file).

Root folder for the assemblies to be loaded:

The path provided can contain environment variables and represents the directory which will be used as the ApplicationBase of the AppDomain that the tests are run in.  All of the assemblies in this directory will be loadable by your tests.  In a production environment a good practice is to set this to the directory where your CUT assemblies are installed.  In a development environment a good practice is to set this to the directory where your CUT assemblies are built to.  This ensures that any references you have to the product binaries can be loaded and resolved during the discovery and execution of the tests without the need to copy the product binaries around with the tests.

If no value is set here, the behavior remains consistent with VS 2008 SP1 and the ApplicationBase of the AppDomain that the tests are run in is set to the directory which contains the tests.

Use the Load Context for assemblies in the test directory:

By default assemblies in the test directory will automatically get resolved (sub directories are not included), and we will ensure that the assemblies get loaded into the “Load Context” when possible (for more info on .Net Load Contexts see links at the end of this post).  For the most part you can leave this checked but there are at least 2 times when you may want to turn this off:

1. If there are a large number of assemblies in your test directory, and you have specified a “Root folder for the assemblies to be loaded”, and your tests are not dependent on being loaded in the Load Context, you may see a performance increase by not using the Load Context to load these test assemblies.
2. If your tests are dependent upon being loaded in a context other then the Load Context (not typical).

Folders to use when the tests are run:

This is the setting that you will likely use the most.  Here you can specify multiple paths to folders that assemblies should be resolved from during discovery and execution of the tests.  Each of the paths specified in this section can contain environment variables.  Along with each of the paths specified here there are 2 options that can be set.  The first option is to use the Load Context when resolving assemblies from this directory (if the load context is not required for the tests to run correctly you may see a performance improvement by unchecking this).  The second option is to include sub folders when resolving assemblies from this directory.

Additional folders to use when discovering tests:

This section is mainly useful when you are either executing the tests remotely under Team Build or doing an automated run from Microsoft Test Manager (MTM).  The paths provided here will be used for assembly resolution, but only during test discovery.  These paths can contain environment variables.  In cases where tests are being scheduled to execute remotely from a build drop and not all of the dependencies of the test assembly are in the same directory, these paths can be used to ensure that MSTest or the Test Controller can find enough of the dependent assemblies to discover the tests and schedule them to the remote machines for execution.

For runs being scheduled from MTM there is an additional token “%BuildDrop%” that can be used to generically refer to the build drop location.  This eliminates the need to create or update a Test Settings each time a new build is being tested.  Unfortunately this token is not directly supported through Team Build (however if the build drop location is set in an environment variable named BuildDrop from the build definition it will have the same result).

In addition to specifying the assembly resolution settings in the Test Settings, you can also put the same type of settings into an App.config file in your test project.  When this is done these settings will only apply to the test project that the App.config file is associated with.  Below is a sample configuration file which will cause assemblies from the “%ProgramFiles%\SampleApplication\” directory to be resolved during test discovery and test execution:

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

  <configSections>

    <section name="TestExecution" type="Microsoft.VisualStudio.TestTools.Execution.TestExecutionSection, Microsoft.VisualStudio.QualityTools.ExecutionCommon, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />

  </configSections>

  <TestExecution xmlns="https://microsoft.com/schemas/VisualStudio/TeamTest/2010">

    <AssemblyResolution>

      <RuntimeResolution>

        <Directory path="%ProgramFiles%\SampleApplication\" includeSubDirectories="true"/>

      </RuntimeResolution>

    </AssemblyResolution>

  </TestExecution>

</configuration>

All of the settings exposed in the UI can be configured in the App.config file as well.  I won’t go through the full schema here but if you edit this file in VS 2010 you will get IntelliSense based on the schema.

One other thing worth mentioning about the settings in the App.config file is that when a Test Settings with assembly resolution settings are used for a test run, the settings in individual tests projects App.config files are overridden.

As promised above here are some Blogs about .Net Load Contexts:

Assembly Load Contexts Subtleties

Choosing a Binding Context

Switching to the Load Context