Introduction to TestApi – Part 1: Input Injection APIs


Series Index

+++

I am starting a series of posts introducing some of the facilities available in TestApi, a test and utility API library, which we recently released on CodePlex. Most of this content is already available in the documentation provided with the library.

The first post is on input injection – a fairly common activity in UI testing.

 

General Notes

Input injection is the act of simulating user input. In general, there are several ways to simulate user input, in the following progressively increasing levels of realism:

  1. Direct method invocation: A test programmatically triggers events by directly calling methods on the target UI element. For example, a test can call the Button.IsPressed method to simulate pressing a WPF button.
  2. Invocation using an accessibility interface (UIA, MSAA, etc.): A test programmatically triggers events by calling methods on an AutomationElement instance that represents the target UI element.
  3. Simulation using low-level input: A test simulates input by using low-level input facilities provided by the host operating system. Examples of such facilities on Windows are the SendInput Win32 API and the Raw Input Win32 API, which inject input directly into the OS input stream.
  4. Simulation using a device driver: A test uses a device driver to simulate input at the device-driver level.
  5. Simulation using a robot: A test controls a robot to simulate direct human interaction with an input device (for example, pressing keys on a keyboard).

Technique A is framework-specific; what works for WPF does not work for Windows Forms and vice versa. Technique B is less framework-specific than A, but still has limitations, because some frameworks differ in their implementations of the required accessibility interfaces. Techniques C and D are OS-specific. Technique D is significantly more difficult to implement and deploy than C, without a corresponding increase in its level of realism. Technique E is universal, albeit much slower and much more expensive than the other options.

The TestApi library provides facilities both for B (through the AutomationUtilities class) and for C (through the Mouse and Keyboard classes), which are the most generally useful techniques of input simulation.

 

 

Examples

The AutomationUtilities class provides wrappers for common UIA operations, such as discovery of UI elements. The first example below demonstrates how to discover and click a WPF Button in a WPF Window, by using the AutomationUtilities class and the Mouse class.

//
// EXAMPLE #1
// This code below discovers and clicks the Close button in an About dialog box, thus
// dismissing the About dialog box.
//

string aboutDialogName = "About";
string closeButtonName = "Close";

AutomationElementCollection aboutDialogs = AutomationUtilities.FindElementsByName(
    AutomationElement.RootElement,
    aboutDialogName);

AutomationElementCollection closeButtons = AutomationUtilities.FindElementsByName(
    aboutDialogs[0],
    closeButtonName);

//
// You can either invoke the discovered control, through its invoke pattern...
//

InvokePattern p = 
    closeButtons[0].GetCurrentPattern(InvokePattern.Pattern) as InvokePattern;
p.Invoke();

//
// ... or you can handle the Mouse directly and click on the control.
//

Mouse.MoveTo(closeButton.GetClickablePoint());
Mouse.Click(MouseButton.Left);

 

The second example below demonstrates how to discover a TextBox instance and type in it, using the Mouse and Keyboard classes as wrappers of common mouse and keyboard operations:

//
// EXAMPLE #2
// Discover the location of a TextBox with a given name.
//

string textboxName = "ssnInputField";

AutomationElement textBox = AutomationUtilities.FindElementsByName(
    AutomationElement.RootElement,
    textboxName)[0];

Point textboxLocation = textbox.GetClickablePoint();

//
// Move the mouse to the textbox, click, then type something
//

Mouse.MoveTo(textboxLocation);
Mouse.Click(MouseButton.Left);

Keyboard.Type("Hello world.");
Keyboard.Press(Key.Shift);
Keyboard.Type("hello, capitalized world.");
Keyboard.Release(Key.Shift);

 

In Conclusion

The Mouse and Keyboard classes in the TestApi library can be used for automating of any application, running on Windows. The classes are completely policy- and context-free – their usage is not dependent on a specific test framework or on a specific test workflow. TestApi provides full source code and XML documentation for these classes, so you can either integrate them in your own projects or reference the pre-built DLLs.

Note that, even though Mouse, Keyboard and AutomationUtilities make the life of the test automation developer quite a bit easier, UI testing is tricky and should be avoided whenever possible. It’s always preferable to design your application as a multi-tier application, with a “thin” UI layer, so that you can bypass the UI in most of your tests.

Comments (15)

  1. I am a bit miffed by the code  EXAMPLE #2

    Concerning,

    Point textboxLocation = textbox.GetClickablePoint();

    … It seems to produce a System.Drawing.Point

    But the next line Mouse.MoveTo(textboxLocation),

    … seems to need a parameter of System.Windows.Point object

    OK… a little conversion…no problem … if the two use the same cooridnate system …

    …it could be i was doing something wrong … if so … please correct…

    thanks,

    Paul

  2. Last week, the WPF test team released a new version of their Test API: 0.2. You can download the new

  3. says:

    到www.CodePlex.com/wpf.看到了TestApi的东东。http://www.codeplex.com/TestApi。好像是用于测试和调试WPF的工具。可能是在程序中实现sn…

  4. Neftali says:

    Can you elaborate on what you mean by  "UI testing is tricky and should be avoided whenever possible. It’s always preferable to design your application as a multi-tier application, with a “thin” UI layer, so that you can bypass the UI in most of your tests." ??

  5. ivom1 says:

    @Neftali,

    Sure! A lot of people these days talk about Model-View separation, using various different patterns (Model-View-Controller, Model-View-Presenter, Model-View-ViewModel, etc.) A central goal of these patterns is to create a system where the Views are not tightly coupled with the Model. This goal has several important implications and one of them is improved testability.

    Imagine for example that you have an IM application. One way to test sending a message is by going through the UI. If you want to do that you will have to (programatically) navigate to the message textbox, type your message, navigate to the Send button, click on it, confirm somehow that your message has been successfully sent. You will have to invest in making the test you write robust so that you can handle different positions of the IM window, intermittent blocks such as another window obscuring your window, different languages and UI positioning, different display resolutions, possibly different OS-es, etc. Some of these will be bona-fide factors you need to include in your test matrix. Others would just be robustness considerations (let's call this "robustness tax"). Now, in order to test your IM message sending functionality completely, you'd have to roll out quite a few variations of the same test — with different message sizes, languages, etc. You'd have to pay this robustness tax in every one of these variations, which can quickly turn into a maintenance nightmare.

    What if you had a way to only test the UI in isolation (without actually sending messages) and only test the message sending functionality (in isolation) without having to deal with UI. Well, this is where Model-View separation comes in handy. Talking to your model without going through the UI is a lot more robust, fast and targeted (it's a method call, not combination of UI interractions AND function call) — you can easily test for a lot of different scenarios that have nothing to do with UI but are potentially bug-prone on the backend. So clearly –  a win, if you can pull that off.

    Hope that elaboration makes sense.

    Ivo

  6. thetpaing says:

    my test program is not wpf. it is normal window

    i have error in

    Mouse.MoveTo(closeButton.GetClickablePoint());

    NonComVisibleBaseClass was detected

    Message: A QueryInterface call was made requesting the class interface of COM visible managed class 'MS.Internal.AutomationProxies.WindowsButton'. However since this class derives from non COM visible class 'MS.Internal.AutomationProxies.ProxyHwnd', the QueryInterface call will fail. This is done to prevent the non COM visible base class from being constrained by the COM versioning rules.

    how to solve this problem?

    thanks.