Interesting problem a customer faced while running x64 based tests through test controller/agent

  • Article

Lot of customers today are running x64 based tests via test controller/agent and things works for them by default. Recently one of our customer recently reported that he is not able to run his tests via test controller/agent and in this post, I want to share out how Raju Ranjan, a developer in our team, debugged & solved this problem so that you can also learn from his experience.

  • The customer mentioned that the run is failing with error similar to this: -

Error adding test case [77979] to test run: Unable to load the test container '\\mymachine\Nightly_Builds\Development2\1.3.0.75\Release\mytest.dll' or one of its dependencies. Error details: System.BadImageFormatException: Could not load file or assembly 'MyTest, Version=1.3.0.0, Culture=neutral, PublicKeyToken=c3309f0734ba2805' or one of its dependencies. An attempt was made to load a program with an incorrect format.

  • On looking at the error, we inferred that the error is happening at the test controller & it is coming because the platform of test assembly does not match with the test controller platform.  Here is how we inferred this.
    • Error adding test case X to run: – Since test controller converts the tfs test run to tmi run (the one which controller/agent understands), it is failing to add the tfs test to the tmi run. So the error is happening at the test controller.
    • System.BadImageFormatException: Could not load file or assembly: – This means that the platform of test assembly or one of its dependency does not match with test controller platform.

 

  • We checked the bit-ness of test controller & the test assembly & its dependencies. We found that the test controller is x86 and one of the dependency is compiled for x64. Also if the dependency is compiled for x86, then the above error does not happen & things work.

 

  • We asked the customer to have 2 test controllers where 1 is x64 based and another is x86 based so that he can run x64 based runs via x64 based controller & x86 based runs via x86 based controller but that was not acceptable to the customer as he had to maintain 2 infrastructures. So we looked for an alternate solution.

 

  • We also knew that the test controller is only doing a discovery of tests from the test binaries as it has to convert only the tfs run to tmi run, so it should be requiring only very minimal dependencies of the test binary.  And if we somehow ensure that the 64 bit dependency is not required to discover the tests, we can make this work via x86 based controller as well.

 

  • We asked the customer to send us the failing solution so that we can check why discovery requires 64 bit dependent binary.

 

  • We built a small utility which does exactly the same thing that we do in discovery i.e. iterating over the types in the test binary and were able to reproduce the problem. The utility is here, just in case you are interested in it as well. The utility outputs all the types which it was able to load. So to find out the types that are causing the problem, we did the following: -
    • Compiled the utility for x64 platform and found out all the types in the test binary.
    • Compiled the utility for x86 platform and found out all the types which the utility could load.
    • Found out the missing types and looked at the code of those types and see the differences.

 

  • We made couple of minor changes in the test code and after that the discovery started working from the x86 process as well. The changes were as follows:
    • Stopped passing “enumeration type” variable defined in the 64 bit dependency to the anonymous delegates and started passing corresponding values. This is required as CLR converts anonymous delegates to classes and attempting to load these anonymous types result in loading of 64 bit dependency.

             Example: – If enumVariable is a variable of type MyEnum, instead of passing enumVariable, we started passing MyEnum.Value1

    • Removed a class level variable/property of a type that is defined in 64 bit dependency and defined it as a “object”

           Example: – If Class Car has a class level variable named m_engine of type Engine where Engine is defined in 64 bit dependency, then changed the declaration from “Engine m_engine” to “Object m_engine” and created a getter/setter methods which typecast it to EngineType and return it.

  • After making the above changes, x86 test controller was able to convert the tfs run to tmi run and things worked.

 

Happy Debugging !!