Extending the Visual Studio Unit Test Type, part 1

  • Article

In Visual Studio 2008, if you wanted to create your own test type, or provide additional functionality that existing test types did not, one would have to create one from scratch.  We provided a sample which was available in the Software Development Kit, but it was somewhat inconvenient and just plain hard to implement.

As part of new functionality that was introduced in the upcoming release of Visual Studio, we have provided the ability to extend the built-in unit test type.

The Finished Product

I want to begin, at the end, asking the question, “What can I do with this extensibility?”

1. The new Coded UI Test is a test type extension:  As you look at what that test type is capable of, then you can imagine that “the sky is the limit”.  You can create your own Sql Server Test Type, or even one that can run testing your use of many other applications such as Oracle®.

2. Add that extra functionality that doesn’t exist in Unit Test Type: You can write a test type that will provide a way to add a row of data and parameterize the test method function, look up the bug database to determine if a caught exception was already filed, or, as we will show in this example, perform impersonation.

Run As Extension : Test Class

This is the end result of our RunAs extension.

image

I have at the top a Traditional Test Class and Test Method.  The test method calls a function that writes to my windows directory (c:\windows).  Since I am an administrator to my machine, the test will pass.

image

My second test, I have a test class but the attribute for the test class is now a "[RunAsTestClass]”.  This will call my test type extension and use the code I have written to execute the test. I create a test property called “RunAsNormalUser” and set it to “true”.  Now, when I run the test, I get a failure.

image

Opening the results, I see that the highlighted text says that Access to the path ‘…’ is denied.

image

In the Debug Trace, we are creating an account called DemoUser.  This account is a standard user.  We then impersonate that user and execute the test.  Since the user does not have permission to write to the c:\windows directory, the test fails, as expected!

On the flip side, we can do exactly the opposite.  We could be running our test framework as a normal user, and then impersonate an administrator.  You would need to pre-create the administrator account, but it is very possible to run tests in as an elevated user.

So lets see what I need to do to implement this.

Run As Extension : References

So with any extension, you need to add the correct references.

There are four assemblies that you need.  Three of them are located in PrivateAssemblies

clip_image001

They are:

  1. Microsoft.VisualStudio.QualityTools.Common.dll
  2. Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel.dll
  3. Microsoft.VisualStudio.QualityTools.Vsip.dll

The last assembly you need is found in the Add References dialog under the .NET tab

clip_image002

Once you have the assemblies referenced it should look something like this:

clip_image003

With your assemblies referenced, you can start creating your files.

Run As Test Type Extension : RunAsTestClassAttribute.cs file

The first file is the RunAsTestClassAttribute.cs file

image

This file does the following:

  1. Inherits from TestClassExtensionAttribute: This is the attribute that will replace the  [TestClass] attribute in our test project (see Run As Extension : Test Class)
    image
  2. Uniquely identifies this attribute with a Uri: We can give it any unique name, in our case, we return “urn:Microsoft.RunAsAttribute”.
    image
  3. Returns the TestExtensionExecution implementation:  This will hold the code we write to make the test do what we want. (see Run As Test Type Extension : RunAsTestExtensionExecution.cs file)
    image
  4. Returns the ClientSide impementation:  This will hold the code we write to provide UI for our test type.  For interest of space on this blog, we will not implement this at this time and will do so in future blogs.
    image

Full code for this class is at the end of the blog

Run As Test Type Extension : RunAsTestExtensionExecution.cs file

The second file is RunAsTestExtensionExecution.cs.  Here is the screenshot of this file.

image

This file does the following:

  1. Inherits from TestExtensionExecution: This allows us to initialize our test and anything that we need for our test type extension to run.  It also provides us a way to cleanup or dispose of any objects or environment items we create during initialize.
    image
  2. Returns the ITestMethodInvoker implementation: This will return a class that will contain the code that will do the impersonation and execute the test (see Run As Test Type Extension : RunAsTestMethodInvoker.cs file)
    image

Full code for this class is at the end of the blog

Run As Test Type Extension : RunAsTestMethodInvoker.cs file

The third file is RunAsTestMethodInvoker.cs.  Here is the screenshot of this file.

image

This file does the following:

  1. Implements from ITestMethodInvoker: This will contain the code that will do the impersonation and execute the test.  We will dive into this code in just a bit.
    image
  2. Contains the constructor for the class that takes a TestMethodInvokerContext:  This gives us information about and a pointer to the actual test being executed.  We can also get test properties, and gather custom attributes from off the function using reflection.
    image
  3. Implements the Invoke method: This is the function that will be called when a test is run and contains the code to impersonate the user and call the underlying test method.
    image
  4. Contains some different member variables we will use in our invoke method.
    image

Full code for this class is at the end of the blog

Lets take a look at the Invoke method

image

  1. First of all, we trace out a bit of information, beginning with the current WindowsIdentity. 
    image
  2. We look at the “RunAsNormalUser” test property which we set back in our unit test and get the value for that property. 
    image
  3. If the property is true, we create an account, using the member variables in the class, and impersonate the user, using some additional classes we have created.  Full code for these classes are at the end of the blog.
    image
  4. We then invoke the method and unimpersonate the user in a try…finally block.
    image

And that is it.

Run As Test Type Extension : Registering your test type extension

To make it all work you now need to place your test type extension assembly in the correct location and register your test type in the registry.

Location:  The location of your test type, and your PDB if you want, needs to be in the PrivateAssemblies location; the same one that contains the assemblies that we referenced in the beginning

clip_image001

Registry: The test type needs to be in the following key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\EnterpriseTools\QualityTools\TestTypes\{13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b}\TestTypeExtensions\DemoTestClassAttribute]

The registry entry is a string called “AttributeProvider” and will contain your attribute and the name of your assembly.
"AttributeProvider"="DemoExtension.RunAs.RunAsTestClassAttribute, DemoExtension"

image

There is a second location that is usually populated when you start Visual Studio.  It is located under the following key: (NOTE the bolded text for the differences in the two keys)

[HKEY_CURRENT_USER\SOFTWARE\Microsoft\VisualStudio\10.0_Config
\EnterpriseTools\QualityTools\TestTypes\{13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b}\TestTypeExtensions\DemoTestClassAttribute]
"AttributeProvider"="DemoExtension.RunAs.RunAsTestClassAttribute, DemoExtension"

I place the second key in manually because sometimes it doesn’t do it for me straight away and I get impatient and cannot wait for the key to be updated.  The full registry file can be found at the end of the blog as RunAsExtension.reg

FINAL REGISTRY NOTE: If you are on an x64 machine, the registry key will be under the Wow6432Node as in HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft…

FINAL LOCATION NOTE: If you are on an x64 machine, the location key will be under the Program Files(x86) folder.

WORKING WITH VS NOTE: Because the test type is loaded within Visual Studio, developing the test type in addition to testing it poses problems when you need to update the test type extension assembly.  To make this work so you can develop in one instance of Visual Studio and test in a separate instance of Visual Studio, you can use the Visual Studio Development Experimental Model found here

Summary

This is just the beginning.  In future blogs we will show additional functionality including implementing a UI and parameterzing your test method.  As it stands, you can start experimenting with the code and add different functionality.  You can initialize anything you want in the TestExtensionExecution and your Invoke method can be customized to suit your needs.

Good Luck!  Let us know if there are any questions or thoughts around this.  Hopefully extending the unit test type will be far easier than doing it from scratch like in Visual Studio 2008.  Doing a test type from scratch is still supported and sometimes needed, but if you can get along by using the test type extension, things will be much easier.

Bruce Taimana
Program Manager
Visual Studio Team Test

Full Code

using System; using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace DemoExtension.RunAs {     [Serializable]     public class RunAsTestClassAttribute : TestClassExtensionAttribute     {         private static readonly Uri m_uri =                    new Uri("urn:Microsoft.RunAsAttribute");

        public override Uri ExtensionId         {             get { return m_uri; }         }

        public override TestExtensionExecution GetExecution()         {             return new RunAsTestExtensionExecution();         }

        public override object GetClientSide()         {             return base.GetClientSide();         }     } }
RunAsTest ExtensionExecution.cs

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace DemoExtension.RunAs {     public class RunAsTestExtensionExecution :                      TestExtensionExecution     {         public override void Initialize(TestExecution execution)         {         }

        public override void Dispose()         {         }

        public override ITestMethodInvoker             CreateTestMethodInvoker(                          TestMethodInvokerContext context)         {             return new RunAsTestMethodInvoker(context);         }     } }
RunAsTestMethod Invoker.cs

using System; using System.Security.Principal; using System.Diagnostics; using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace DemoExtension.RunAs {     public class RunAsTestMethodInvoker : ITestMethodInvoker     {         private TestMethodInvokerContext m_context;

        public RunAsTestMethodInvoker(                          TestMethodInvokerContext context)         {             Debug.Assert(context != null);             m_context = context;         }

        public TestMethodInvokerResult Invoke(params object[] args)         {             // Log the ID of the test user             Trace.WriteLine("Begin Invoke: current user is " +                               WindowsIdentity.GetCurrent().Name);             RunProcessAs runas = null;

            bool runAsNormalUser;             Boolean.TryParse(                        m_context.TestContext.Properties["RunAsNormalUser"]                        as string, out runAsNormalUser);

            if (runAsNormalUser)             {                 Trace.WriteLine("Creating user: " + USER);                 UserAccounts.CreateUserInMachine(USER,                           PASSWORD, UserAccounts.GroupType.Users);

                // Impersonate a user with minimal privileges                 Trace.WriteLine("Impersonating user: " + USER);                 runas = new RunProcessAs(USER, DOMAIN, PASSWORD);             }

            // Invoke the user's test method             Trace.WriteLine("Invoking test method");

            try             {                 return m_context.InnerInvoker.Invoke();             }             finally             {                 if (runas != null)                 {                     // Undo the impersonation                     Trace.WriteLine("Undoing impersonation of user: "                           + USER);                     runas.Dispose();

                    Trace.WriteLine("Removing user: " + USER);                     UserAccounts.RemoveUserFromMachine(USER);                 }

                // Log the ID of the test user                 Trace.WriteLine("End Invoke: current user is "                         + WindowsIdentity.GetCurrent().Name);             }         }

        private const string USER = "DemoUser";         private const string DOMAIN = "";         private const string PASSWORD = "abc123!!";     } }
RunAs.cs

using System; using System.ComponentModel; using System.Runtime.InteropServices; using System.Security.Principal; using System.Threading;

namespace DemoExtension.RunAs {     public enum LogonType     {         Interactive = 2,     }

    public enum LogonProvider     {         Default = 0,     }

    public sealed class RunProcessAs : IDisposable     {         WindowsImpersonationContext impersonationContext = null;

        public RunProcessAs(string user, string domain, string password)         {             Impersonate(user, domain, password,                      LogonType.Interactive, LogonProvider.Default);         }         public RunProcessAs(string user, string domain,                       string password, LogonType logonType,                       LogonProvider logonProvider)         {             Impersonate(user, domain, password,                      logonType, logonProvider);         }         ~RunProcessAs()         {         }         public void Dispose()         {             if (constructorThread != Thread.CurrentThread)             {                 throw new ApplicationException(                       "Dispose should be called on the same thread as instance constructor.");             }             Exception ex = null;             if (IntPtr.Zero != token)             {                 impersonationContext.Undo();                 if (!NativeMethods.CloseHandle(token))                 {                     ex = new Win32Exception(Marshal.GetLastWin32Error());                 }                 token = IntPtr.Zero;             }             GC.KeepAlive(this);             GC.SuppressFinalize(this);             if (ex != null) throw ex;         }         private void Impersonate(string user, string domain,                     string password, LogonType logonType,                      LogonProvider logonProvider)         {             if (null == user) throw new ArgumentNullException();             if (null == domain) throw new ArgumentNullException();             if (null == password) throw new ArgumentNullException();             //             if (!NativeMethods.LogonUser(user, domain, password,                       logonType, logonProvider, out token))             {                 throw new Win32Exception(Marshal.GetLastWin32Error());             }

            impersonationContext = WindowsIdentity.Impersonate(token);             if (impersonationContext == null)             {                 NativeMethods.CloseHandle(token);                 token = IntPtr.Zero;                 throw new Exception("Failed to impersonate specified user");             }

            constructorThread = Thread.CurrentThread;             GC.KeepAlive(this);         }         private IntPtr token = IntPtr.Zero;         private Thread constructorThread = null;     }

    abstract class NativeMethods     {         [DllImport("advapi32.dll", SetLastError = true)]         internal extern static bool LogonUser(string user,                      string domain, string password, LogonType logonType,                      LogonProvider provider, out IntPtr token);         [DllImport("kernel32.dll", SetLastError = true)]         internal extern static bool CloseHandle(IntPtr handle);         [DllImport("advapi32.dll", SetLastError = true)]         internal extern static bool ImpersonateLoggedOnUser(IntPtr token);         [DllImport("advapi32.dll", SetLastError = true)]         internal extern static bool RevertToSelf();

    } }
UserAccounts.cs

using System; using System.Text; using System.Reflection; using System.Text.RegularExpressions; using System.Collections; using System.Collections.Generic; using System.Collections.Specialized; using System.Xml; using System.IO; using System.Globalization; using Microsoft.Win32; using System.Diagnostics; using System.Security.Permissions; using System.DirectoryServices; using System.Security.Principal;

namespace DemoExtension.RunAs {     public class UserAccounts     {         public enum GroupType         {             Users         };         static string[] strGroupType = { "Users" };

        static string[] sidGroupType = { "S-1-5-32-545" };

        //This is for localization runs where Administrators         //   might be Administrateurs etc         static int findIndexForGroupType(string groupType)         {             for (int i = 0; i < strGroupType.Length; i++)             {

                if (strGroupType[i].ToLower().Equals(                                 groupType.ToLower()))                 {                     return i;                 }             }             return -1;         }

        public static string GetLocalGroupString(string groupType)         {

            int index = findIndexForGroupType(groupType);             if (index < 0)                 throw new ArgumentException("groupType");             SecurityIdentifier sid =                  new SecurityIdentifier(sidGroupType[index]);             NTAccount ntaccount =                  sid.Translate(typeof(NTAccount)) as NTAccount;

            string[] accountTokens =                  ntaccount.ToString().Split(new char[] { '\\' });             switch (accountTokens.Length)             {                 case 2:                     return accountTokens[1];                 case 1:                     return accountTokens[0];                 default:                     throw new Exception(                     "Account Token not in the known format");             }

        }

        public static void CreateUserInMachine(string User,                       string Password, GroupType groupType)         {             InternalCreateUserInMachine(User, Password, groupType);         }         static void InternalCreateUserInMachine(string User,                        string Password, GroupType groupType)         {             try             {                 InternalRemoveUserFromMachine(User);             }             catch { }

            DirectoryEntry AD = new DirectoryEntry("WinNT://" +                                 Environment.MachineName + ",computer");             DirectoryEntry NewUser = AD.Children.Add(User, "user");             NewUser.Invoke("SetPassword", new object[] { Password });             NewUser.Invoke("Put", new object[] {                              "Description", "Test User from .NET" });             NewUser.CommitChanges();

            DirectoryEntry grp;

            grp = AD.Children.Find(GetLocalGroupString(                          strGroupType[(int)groupType]), "group");             if (grp != null) { grp.Invoke("Add", new object[] {                          NewUser.Path.ToString() }); }

        }

        public static void RemoveUserFromMachine(string User)         {             InternalRemoveUserFromMachine(User);         }

        static void InternalRemoveUserFromMachine(string User)         {             try             {                 DirectoryEntry AD = new DirectoryEntry("WinNT://" +                                     Environment.MachineName + ",computer");                 DirectoryEntry UserToRemove =                           AD.Children.Find(User, "user");                 AD.Children.Remove(UserToRemove);             }             catch (Exception ex)             {                 throw ex;             }         }     } }
RunAsExtension.reg (NOTE: Remove any line breaks in the registry key, name or value)

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\ 10.0\EnterpriseTools\QualityTools\TestTypes\ {13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b\TestTypeExtensions \DemoTestClassAttribute] "AttributeProvider"= "DemoExtension.RunAs.RunAsTestClassAttribute,         DemoExtension"

[HKEY_CURRENT_USER\SOFTWARE\Microsoft\VisualStudio\ 10.0_Config\EnterpriseTools\QualityTools\TestTypes\ {13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b\TestTypeExtensions \DemoTestClassAttribute] "AttributeProvider"= "DemoExtension.RunAs.RunAsTestClassAttribute,         DemoExtension"