Custom UI Automation Providers in Depth: Part 1

Creating a UI Automation provider for a custom control is not a difficult task. At least, we members of the UI Automation team don’t think so. Yet, it is one that engineers continue to find challenging. And so it seems worth writing a walkthrough. I’ll also post the full code samples on the MSDN Code Gallery so that you can explore and re-use them.

I am Michael Bernstein, the development lead for the Accessibility team in Windows. I want to start with a quick word about the philosophy of what we’re trying to do here. Programmatic accessibility is really all about taking a user interface and turning it into a programming interface. I’m sure you’ve all used programmatic interfaces of various kinds: library functions, APIs, web services, etc. So, how would you query a word processing program for data – like you can do with a database? Or how would you issue commands to a Control Panel – as you would to a web service? The problem is, the Control Panel or word processor doesn’t have anything to code against. So, that’s the problem we’re trying to solve.

We’ll need a particular example for our walkthrough. Most of the standard UI controls, like buttons and combo boxes, already have UI Automation interfaces. But a custom control would not. So, I created a custom control: the TriColor control. It’s a bit silly, but bear with me. The TriColor control lets you select one of three colors: Red, Yellow, or Green. It would be fabulous for an application that controlled a traffic light. Clicking on a color moves the selection, and it has some basic keyboard support for the up and down arrows to do the same thing. It’s not a complicated control, but it’s good enough to demonstrate the concepts here.

original[1]

We’ll also need something to test how we’re doing. One of the most classic Accessibility tools is the screen reader, but using a screen reader for testing adds a layer between you and the programmatic interface you are creating. You’d like to see the interface itself. The tool our team uses most often is called Inspect Objects, and you can find it in the Windows 7 Platform SDK. So, if you don’t have that SDK, download it and make sure you can run the Inspect tool. The Inspect tool defaults to following both focus and cursor position, so hover your mouse over anything to inspect its properties. The menu item Options | Show Highlight Rectangle is very useful to show you exactly what you are inspecting. If you haven’t tried this before, inspect various parts of your desktop and see what kinds of properties show up.

Experiment: Go to the MSDN Code Gallery site and get the UIAControls-Part1.zip file.  Compile the TricolorControl program, run it, and inspect it with Inspect. What do you see?

Here’s what I see:

original[1]

When I do it, I notice that even though I have done nothing with UI Automation yet, I do see a few Accessibility properties:

  • The Control Type is ‘pane.’ (That just means a container.)
  • The bounding rectangle is accurate.
  • The RuntimeID property is there. (It’s correct, too.)
  • The IsKeyboardFocusable and HasKeyboardFocus properties are both true. (If the program has focus.)

Well, that’s a good trick – I got UIA properties for free! How did that happen? Well, UIA can see that the control has a window handle, and the window itself has a bunch of properties. UIA picks these up as defaults. This is convenient – it saves you the trouble of providing them. (Although you can override them, as we’ll see.) The defaults are provided by the HWND provider, which exists for every control with a window handle.

However, we’re missing a whole bunch of information:

  • What is the control’s name?
  • The ControlType is wrong – this isn’t just a window pane. What is it really?
  • What value is currently set (Red, Yellow, Green)?
  • Where are the red, yellow, and green regions? How do they relate to the whole control?
  • Can I change the value?

To answer these questions, we need to extend the control’s programmatic interface. And we’ll do that by implementing a UI Automation for the control, in the next post.

One last word about language: my sample uses C# and WinForms. WinForms is close enough to the Windows native APIs that the techniques I show here can be translated easily into C++. But the syntactic sugar of C# makes the examples shorter and easier to follow. When I’m doing something in C# that doesn’t translate easily in C++, I’ll point it out. I’m also not using WPF. WPF has a different system: it does a bunch of UI Automation work on behalf of regular controls and even custom controls. This is great – go WPF! – but it would undermine what I’m trying to demonstrate here. The MSDN content on implementing UI Automation in WPF is quite good, so read that if that’s your framework.

Next time: Creating the “Hello world!” of UI Automation providers.