Extending the Visual Studio Unit Test Type, part 2

  • Article

In part 1, we introduced Unit Test Type Extensibility, highlighting the main parts of the interface.  In this round, we will discuss three additional pieces:

  1. Adding a user interface
  2. Parameterizing the test method
  3. Aggregating test results
  4. UI extension

Due to the amount of code in this sample, I will paste some of it at the end of the blog.  The rest I will try to upload as a file to the blog.

The Finished Product

Lets begin again at the end.  “What do these three additional pieces give me?”

A snapshot seems appropriate here

image

Yes, I know, unit test should already have this built into the standard implementation, but since we have not done this, one is able to build a very quick implementation of it for your own testing.

Lets go through this code.

  1. The RowTestClass attribute tells the framework to load my test type extension to run these tests
    image
  2. We have a new custom attribute, called Row, that will allow me to pass in the data that I want to send to my parameterized test method
    image
  3. Now, with our test method being parameterized, we can use those variables directly in our test method instead of having to pull them off the DataRow in the TestContext
    image

Lets blaze on ahead with the different files.  I will quickly show the References, TestClassAttribute, TestExtensionExecution, and parts of the Invoker.  We will dive deeper into the invoker, aggregation and UI code.

Row Test Test Type Extension : References

First we need the references.  The Microsoft.VisualStudio.QualityTools.UnitTestFramework assembly can be found in the Global Assembly Cache, and the other three Microsoft assemblies can be found in PrivateAssemblies.  For more information, see Extending the Visual Studio Unit Test Type, part 1: Run As Extension : References section

 image

Row Test Test Type Extension : RowTestClassAttribute.cs file (quick view)

File number 1, RowTestClassAttribute.cs, is similar to the RunAsTestClassAttribute with the following differences:

image

  1. The Uri is unique to this attribute.
  2. In our GetClientSide implementation, we return a RowTestExtensionClientSide object. (see Row Test Test Type Extension : RowTestExtensionClientSide.cs file)

For more information, see Extending the Visual Studio Unit Test Type, part 1 for more information regarding the TestClassExtensionAttribute.

Row Test Test Type Extension : RowTestExtension.cs file (quick view)

File number 2, RowTestExtension.cs, is similar to the RunAsTestExtensionExecution.cs file and there only difference is the object type returned, which is a RowTestInvoker class (see Row Test Test Type Extension : RowTestInvoker.cs file (quick view) )

image

Row Test Test Type Extension : RowTestInvoker.cs file (quick view)

File number 3, RowTestInvoker.cs, is similar to the RunAsTestMethodInvoker.cs, with the difference being in the Invoke method.

image

The Invoke method is where it gets interesting.

  1. First we initialize our RowTestResults class.  This class is used to aggregate our results when we execute them.
    image 
  2. We then gather all the RowAttributes off the test method using reflection.image
  3. We then check the count of the RowAttributes we have.  If we don’t have any, we will invoke the base method passing a null as a parameter.
    image
  4. If we do find a list of RowAttributes, we will iterate through them and call the Invoke method, passing the row information.  We also aggregate them into our the RowTestResults instance that we created in step #1.
    image
  5. Finally we return the single aggregated results
    image

Row Test Test Type Extension : RowTestExtensionClientSide.cs file (quick view)

This file allows us to return the UserControl which will be displayed in the UI when the results details is requested.  It also has the icon we would like to use for our Test Type Extension.image

The Icon property returns an embedded Icon that I placed in the Resources for the project.

The ResultExtensionViewer returns the user control that we have created to display our results.

Row Test Test Type Extension : RowTestExtensionViewer.cs file (quick view)

This is a user control, so it also has a .Designer.cs and a .resx file with it.  However, the most important is the implementation of the User control.  It must implement ITestTypeExtensionResultViewer

image

Of note, the Initialize function is what makes this work.

  1. The initialize takes a TestResult argument.  This is will contain quite a bit of information regarding outcome, error, stack trace and other items you may need for that result.  But keep in mind, this is the overall result.  You can call it the result container.
    image 
  2. For our purpose we need to get a little piece of information off it regarding the ITestResultExtension.  So we cast it, check that it isn’t null and dump the results to it (since we returned a string earlier (see Row Test Test Type Extension : RowTestInvoker.cs file – Step #5).
    image
    This line also assumes that we have placed a label on our UserControl.

And that is it.  A little more work but now we are ready to deploy

Row Test Test Type Extension : Registering your test type extension

To make it all work you now need to place your test type extension assembly in the correct location and register your test type in the registry.

Location:  The location of your test type, and your PDB if you want, needs to be in the PrivateAssemblies location; the same one that contains the assemblies that we referenced in the beginning

clip_image001

Registry: The test type needs to be in the following key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\EnterpriseTools\QualityTools\TestTypes\{13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b}\TestTypeExtensions\RowTestClassAttribute]

The registry entry is a string called “AttributeProvider” and will contain your attribute and the name of your assembly.
"AttributeProvider"="DemoExtension.RowTest.RowTestClassAttribute, DemoExtension"

image

There is a second location that is usually populated when you start Visual Studio.  It is located under the following key: (NOTE the bolded text for the differences in the two keys)

[HKEY_CURRENT_USER\SOFTWARE\Microsoft\VisualStudio\10.0_Config
\EnterpriseTools\QualityTools\TestTypes\{13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b}\TestTypeExtensions\RowTestClassAttribute]
"AttributeProvider"="DemoExtension.RowTest.RowTestClassAttribute, DemoExtension"

I place the second key in manually because sometimes it doesn’t do it for me straight away and I get impatient and cannot wait for the key to be updated.  The full registry file can be found at the end of the blog as RowTestExtension.reg

FINAL REGISTRY NOTE: If you are on an x64 machine, the registry key will be under the Wow6432Node as in HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft…

FINAL LOCATION NOTE: If you are on an x64 machine, the location key will be under the Program Files(x86) folder.

WORKING WITH VS NOTE: Because the test type is loaded within Visual Studio, developing the test type in addition to testing it poses problems when you need to update the test type extension assembly.  To make this work so you can develop in one instance of Visual Studio and test in a separate instance of Visual Studio, you can use the Visual Studio Development Experimental Model found here

Summary

Good Luck!  Let us know if there are any questions or thoughts around this.  Hopefully extending the unit test type will be far easier than doing it from scratch like in Visual Studio 2008.  Doing a test type from scratch is still supported and sometimes needed, but if you can get along by using the test type extension, things will be much easier.

Bruce Taimana
Program Manager
Visual Studio Team Test

Full Code

We now have our SDK available and code can be found here when it goes live. 

The download is found on code gallery here.