Testing WPF applications with the White UI Test framework

Article
04/03/2008

For the past few weeks, I have been test driving the White UI Test framework. It is an open source extension for the NUnit test framework. There are already blog tutorials on using the White UI Test framework with WinForms (see Ben Hall's blog) so I focused on using it to test WPF applications. The only minor constraint I have found is that White requires .Net 3.0 which is a large download and the download might be an issue for some VS 2005 users. I also recommend that you install the Windows SDK so that you have the UI Spy tool and the samples.

The premise of the tool is to use the System.Windows.Automation namespace to access UI pages and controls for test purposes. Numerous developers have written individual tests using the System.Windows.Automation namespace (see for example this blog post) but White is the first reusable library of useful namespaces and methods.

There are advantages to using the System.Windows.Automation. It gives the tool versatility so that it can support testing Win32, Win Form, WPF and SWT (java) applications. Since the tool can access anything the Automation API’s find, it may be possible to automate tests for Visual Studio add-ins. I verified that the Automation API’s can find windows for Visual Studio add-ins but have not attempted it. Another advantage is that I haven’t had to create NUnit configuration files as I reported when I tried writing WPF tests from “scratch.”

Here’s a full example using a simple WPF application. The application is a basic master list and detail scenario. It uses some data binding techniques. If you’re new to WPF, you can learn a lot about data binding by reading Bea Costa’s blog. There are many useful examples there.

The application allows the user to add people to a list. The Person object it uses contains only the person’s name and gender. So, there is a TextBox control to enter the name, a ComboBox which is bound to the Gender enum type defined in the application, and a button to actually add the person. The People class is an ObservableCollection of Person objects and it is databound to a ListBox control. The list displays only the name of each Person object. When the user selects a name from the list, the person’s name and gender are displayed in Label controls in the detail block at the bottom of the window.

Assuming that you have NUnit and White installed, you need to define references to NUnit.Framework, Core (the main White DLL), and White.NUnit, the DLL which contains White’s Verify class. The Verify class does stop the test if the results are wrong so you can find a bunch of errors in one run. At the end of the test, you have to check that none of the Verify statements failed. You also need a reference to the namespace of the application you are testing. For convenience, I usually define those references and add additional using statements so that I end up with something like this:

// Additional using statements

using NUnit.Framework;

using Core;

using Core.UIItems;

using White.NUnit;

using ListBoxExample;

The first thing the test has to do is start the application you want to test and find the main WPF window. Before I go too far into the example, I want to say that there is very little difference in testing WPF versus WinForms applications. Basically, you have to change the type of a few White objects but the logic would be essentially the same. So here’s how you use White’s Application class to start the application and find the UI window:

// Start the app

Application application = Application.Launch("ListBoxExample.exe");

Assert.IsNotNull(application);

// Find the main window

window = (Core.UIItems.WindowItems.WPFWindow)application.GetWindow("Window1", Core.Factory.InitializeOption.NoCache);

Assert.IsNotNull(window);

You can do most UI actions using White’s controls. However, there may be occasions when you need keyboard input. White has two classes for doing that. However, you should use the AttachedKeyboard class. The Keyboard class is primarily used by the AttachedKeyboard class. Here’s how to use the AttachedKeyboard class to add a Person object to the application’s list:

// You access the keyboard via the Keyboard property of the window that has focus

Core.InputDevices.AttachedKeyboard keyboard = window.Keyboard;

// Tab to the name textbox

keyboard.PressSpecialKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.TAB);

// Type the name into the name text box

keyboard.Enter("Harriet");

// Tab to the combobox

keyboard.PressSpecialKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.TAB);

// Type a g into the combo box to select Girl

keyboard.Enter("g");

// Tab to the add button control

keyboard.PressSpecialKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.TAB);

// Hit the Enter key

keyboard.PressSpecialKey(Core.WindowsAPI.KeyboardInput.SpecialKeys.RETURN);

That’s doing it the hard way. Here’s the easy way to add people using White objects:

// Add a person using White controls

Core.UIItems.ListBoxItems.WPFComboBox comboBox = window.Get<Core.UIItems.ListBoxItems.WPFComboBox>("genderComboBox");

Assert.IsNotNull(comboBox);

TextBox nameTextBox = window.Get<TextBox>("nameBox");

Assert.IsNotNull(nameTextBox);

Button button = window.Get<Button>("addButton");

Assert.IsNotNull(button);

string[] names = { "Tom", "Dad", "Mom", "Jane", };

Genders[] genders = {Genders.Boy,Genders.Man,Genders.Woman,Genders.Girl, };

for (int i = 0; i < names.Length; i++)

{

nameTextBox.Text = names[i];

comboBox.SetValue(genders[i]);

button.Click();

}

Now, the test will need to select an item from the list and check the detail box to be sure the information is correct. You can select items by either their index in the list or the text that the object’s ToString method returns:

// Find the ListBox by the control’s name

Core.UIItems.ListBoxItems.ListBox theList = window.Get<Core.UIItems.ListBoxItems.ListBox>("theListBox");

// Select the fourth item in the list by index; Like arrays, the index is from 0 to Length-1

theList.Select(3);

// Find the detail info for the selected item (name and gender)

Label selectedItemNameLabel = window.Get<Label>("selectedItemName");

Assert.IsNotNull(selectedItemNameLabel);

Label selectedItemGenderLabel = window.Get<Label>("selectedItemGender");

Assert.IsNotNull(selectedItemGenderLabel);

// The data should be Mom and Woman

Verify.AreEqual("Mom", selectedItemNameLabel.Text);

Verify.AreEqual(Genders.Woman.ToString(), selectedItemGenderLabel.Text);

// Select an item by the object's ToString text

theList.Select("Harriet");

// The data should be Harriet and Girl

Verify.AreEqual("Harriet", selectedItemNameLabel.Text);

Verify.AreEqual(Genders.Girl.ToString(), selectedItemGenderLabel.Text);

At the end of the test, you need some logic to see if any of the Verify method calls failed and print any error messages. I have asked the project owner to add a method to the Verify class that will do this and he has agreed but, at the time I wrote this blog, the new method was not yet available. Here’s what you can do in the mean time:

// The Verify class has a bool property that tells you if any call to Verify failed

// If Verify.TestFailed is true, this logic prints error message to stdout and asserts

Console.WriteLine();

if (Verify.TestFailed == true)

{

// The Failures property is a collection of error messages from the test's failures

foreach (string err in Verify.Failures)

Console.WriteLine(err);

// Throw an expception and tell the user where to find the error messages

Assert.Fail ("Test encountered " + Verify.Failures.Count +

" failures. See Output Tab for listing");

}

Along with this post, I have uploaded a zip file with the example outlined above. It also has an additional “person” and some incorrect calls to Verify methods so that you can see the detail level that White provides on its error messages. The zip file can be found as an attachment at the end of this post.

I like the White tool. However, there are some disadvantages to using the System.Windows.Automation namespace:

Since reflection is not used, the test cannot access information about the application’s objects bound to a WPF control. Suppose you had an application that had a master/detail pattern in its UI. You might want to write a test that verifies that the detail display is actually from the object selected in the master list. This is not possible without using reflection.

Consider this XML snippet:

<Label>This label does not have a name.</Label>

</StackPanel>

The System.Windows.Automation namespace only provides one control type for text information. WPF provides three controls, TextBlock, Label, and TextBox. Since a WPF TextBox control allows input and the others are read-only, White can distinguish TextBox controls from the other two but cannot determine whether the developer used a TextBlock or Label control. You can find those text controls with White by either the control’s name or its text. Here’s a code snippet that shows how to do it:

application = Core.Application.Launch("BasicTextBlock.exe");

window = (Core.UIItems.WindowItems.WPFWindow) application.GetWindow("Window1", Core.Factory.InitializeOption.NoCache);

Label aDisplayedLabel = (Label)window.Get<Label>("someTextBlock");

Assert.IsNotNull(aDisplayedLabel, "Could not find someTextBlock by Name");

aDisplayedLabel = (Label)window.Get<Label>(SearchCriteria.ByText("This label does not have a name."));

Assert.IsNotNull(aDisplayedLabel, "Could not find Label by its Text");

TextBox textInputBox = window.Get<TextBox>("anInputTextBox");

Assert.IsNotNull(textInputBox, "Could not find TextBox by Name.");

System.Windows.Automation namespace gives the user access only to a displayed object’s text field. To use display collections of objects in a list or tree view, an application must provide a ToString override for the object’s class which provides some unique information or all objects in the collection will be reported as the object’s type.

System.Windows.Automation namespace cannot find TextBlock controls in WPF DataTemplates but does find Label controls. There have been other reports of the System.Windows.Automation namespace having difficulties with WPF DataTemplates. Here’s a screenshot showing what I’m talking about:

Here’s an XML snippet showing the DataTemplate’s definition:

<Label FontWeight="Bold">This ContentControl detail block uses Label and TextBlock controls defined in a DataTemplate.</Label>

<Label FontWeight="Bold">UI Spy is able to find the Labels but not the TextBlock controls.</Label>

<Label FontWeight="Bold" Target="{Binding ElementName=OrbitBlock}">Orbit:</Label>

</StackPanel>

</StackPanel>

</StackPanel>

</StackPanel>

</Border>

</DataTemplate>

We use the DataTemplate like this…

This XML snippet shows similar text controls defined in line:

<Label FontWeight="Bold">This ContentControl detail block uses Label and TextBlock controls defined in line.</Label>

<Label FontWeight="Bold">UI Spy is able to find both the Label and the TextBlock controls.</Label>

</StackPanel>

</StackPanel>

</StackPanel>

</StackPanel>

</Border>

</ContentControl>

That’s about it for now. If I have time, I will try testing a VS add-in with White and let you know what I find.

ListBoxExample.zip

Testing WPF applications with the White UI Test framework

Additional resources