Understanding the Window Search and Windowed Properties

  • Article

There has been questions regarding how do you search a particular control giving ControlName or ControlId etc. So I thought it is a good opportunity to blog about it. I will be covering the following :-

a) Which plugin gets picked up for which controls

b) Introducing MSAA plugin

c)  Introducing what does it mean that a control is windowed.

d) Properties specific to Windowed control.

e) How to do Windowed based search and how to search a particular non windowed control using a windowed properties.

Plugins currently picked up (Beta2)

Below is the info in which plugins gets picked up for which controls

1) IEPlugin (web) :- This is picked up for web controls inside IE 7/8.

2) UIA:- This is picked up for WPF controls.

3) MSAA (Default plugin) :- This is picked for Winforms Controls, Win32 controls, MFC applications. In short which ever controls that are not specified in above two plugins gets picked up by this plugin which is a default plugin. Thing to note here is that if for some platform we are sure that default plugin wont work (for eg. FireFox, SIlverlight etc) this plugin also doesn’t get picked up and we throw exception.

 

Introducing MSAA plugin

MSAA plugin uses “Micorsoft Active Accessibility” to identify controls. So if a control has a good Accessibility support then it could be identified by the plugin (if there is no other plugin that support that control this plugin is hit. To see how to write your own plugin for the platform that are not supported please go through this.)

Internally the plugin generates search properties/exposes properties given by interface IAccessible. For richer properties for Winforms/Win32 controls it uses Windows messages to get those properties. Also some of the SearchProperties are windowed specific properties. In the next couple of topic we will be seeing what does a windowed control means and what are the specific properties that this plugin generate.

 

What are windowed controls?

By Windowed controls we mean all the controls who have their own window handle. You can identify if a particular control has a window handle of it’s own by using SPY++. Below are the steps to check if a control has a window handle of it’s own or not.

1) Open SPY++.

2) Press Ctrl+F.

3) Use the Cross hair and drag it to task bar button of any application that is opened. You will notice that the whole task bar is getting highlighted instead of the buttons. This is because it uses a window handle from point and highlights it. In this case the buttons don’t have their own window handle instead the whole task bar’s window. This means all the these buttons inherits the window handle of the parent task bar window control.

Note: Not all controls with ControlType = Window are windowed and also even a control which is of ControlType other than Window can also be a Windowed control (eg the hosting control inside the Internet Explorer who’s controltype is a client)

Properties of Windowed Control

Some the properties that a windowed control can have are :-

1) WinProperties.Common.ControlName (The ControlName which is got by Sending WM_GETCONTROLNAME message to the window handle of the control)

2) WinProperties.Common.ControlId (Got using GWL_ID message)

3) WinProperties.Common.ClassName (Got using GetClassName Pinvoke)

4) WindowText (Got using GetWindowText PInvoke)

5) AccessibleName (Got using IAccessible pointer for the control)

All of these properties are got using window handle of the control and hence these are specific to windowed controls. The last property though (as the name suggest is not a property got using window handle but using IAccessible pointer.

How is Name and AccessibleName different?

With Respect to the MSAA plugin Name is IAccessible name for a non windowed control. In case of Windowed controls the name Property returns WindowText and AccessibleName returns IAccessible Name. For Non Windowed controls AccessibleName property should not return anything and you will get IAccessible name using Name property as mentioned before.

 

Searching of a control

You can do two type of Search using MSAA plugin :-

1) Window based search :- This is done by navigating through all the window either inside Desktop or inside a specified Window handle.

2) MSAA search :-This uses IAccessible based navigation to search a particular control.

 

If you want to search a window control you can do that by specifying atleast one Window Property (as mentioned above)

So you can do this to search a control who’s ControlName is “foo” and who’s controltype is window

WinWindow window = new WinWindow (containerelement);
window.SearchProperties.Add(WinProperties.Common.ControlName, "foo");

Internally the plugin checks if there is even a single window property mentioned in SearchProperty. Seeing that it does a Window based search.

Note: Following are the properties you can mention as part of SearchProperties so that a window based search kicks in.

WinProperties.Common.ControlName

WinProperties.Common.ControlId

WinProperties.Common.ClassName

“AccessibleName”

 

Searching a Non Windowed control using Windowed property

Let’s take an example of a Simple Windows Form Application which has a button inside it. I want to search the button if I know it’s parent window’s ControlName (let’s say “foo”) and don’t want to search using button’s name property. Below code will show how to search a button using it’s window’s controlname property

WinWindow window = new WinWindow (topLevelElement);

// Assign a ControlName property to the window control so that a window search kicks in.
window.SearchProperties.Add(WinProperties.Common.ControlName, "button1");

// Now you search a button control without specifying any property here

WinButton button = new WinButton (window);
Mouse.Click(button);

Let’s understand the code. The window instance is having a Windowed SearchProperty assigned to it. Hence it will hit a window search which matches the control name property before returning the element.

Now we have assigned the Container element as window of the button control (see the ctor of the button control). So internally a search for the window will happen first using window search  and then button will be searched under the window control using MSAA based search.