Role-Based Testing Strategy for Verifying Microsoft Active Accessibility

  • Article

Below is something I’ve been working on for a while now, putting all of my thoughts regarding MSAA testing down onto paper.  Consider this the grand specification for MsaaVerify.  Provided i had enough time, this is everything i wanted to implement in MsaaVerify.  Hopefully someone out there will find this useful and can put it to use. 

Role-Based Testing Strategy for Verifying Microsoft Active Accessibility

This testing strategy assumes a UI control has correctly identified itself via a MSAA property called a role.  For example, an OK button on a dialog has a MSAA property called “Push Button”.  Based on the control’s role, a specific suite of tests for that particular control is run.  These tests verify that the control has implement its other properties and controls correctly, based on its supplied role.

For standard Windows controls, like a Push Button or an Edit Box, this verification is a straight-forward process. See https://msdn.microsoft.com/library/en-us/msaa/msaapndx_2a05.asp for examples.  But for more complex controls, where 2 controls merge, or new controls are created, the verification hasn’t been defined yet. 

Section 1 – Defining Test Cases

This first section is how to define the test cases.

  1. Run test suites against a UI control based on its MSAA role, defined by IAccessible::get_accRole.  The role is one of the standard Windows controls.  See https://msdn.microsoft.com/library/en-us/msaa/msaapndx_6ypa.asp for an example of how to implement MSAA for a push button.
  2. Combine test suites against a UI control, if the control’s MSAA role is a “hybrid”.  A hybrid control is where 2 standard Windows controls are combined to form one control.  For example, a Checked Tree View is a hybrid of a check box and a tree view.  To combine test suites, the more important verification for each property is used over the other.  For example, a check box does not implement IAccessible:;get_accValue; however, for a tree view item, IAccessible:;get_accValue returns the depth in the tree.  Hence for a Checked Tree View, the MSAA Value property test verifies that the correct depth in the tree is returned.
  3. Copy test suites against a UI control, if the control is a “clone” of some other standard Windows control.  A clone is a control that is completely new, but mimics the behavior of a standard Windows control.  For example, a developer might implement some snazzy new controls that resembles a floating balloon.  Whenever this control is clicked, a “tip of the day” appears.  Take away the eye candy, the control is really just a glorified push button.  Therefore, the first test suite must verify that the clone’s MSAA role is “Push Button”.  Then, the “Push Button” test suites are run.
Section 2 – Tweaking the Testing Intensity

Just like in any other aspect of software testing, there’s a dial that a software tester can turn up or turn down the amount or intensity of the applied testing.  The dial for testing MSAA can be viewed at the following 4 settings:

Dial Setting #1: Strictest level of testing

All expected method and properties for given role (as defined by the test suite) must be implemented and implemented correctly.  Anything else implemented is considered a failure.

Consider an OK button on a dialog. 

  1. The MSAA Default Action property for the button is “Press”.  If this method is not implemented correctly, ie “Click” is returned, or if this method isn’t implemented at all, a failure is reported
  2. The MSAA Value property for the button is “OK”.  A push button is not supposed to implement the Value property.  A failure is reported.
  3. The MSAA Name property for the button is not implemented.  All UI controls must have a name, so this property must be implemented.  A failure is reported.
  4. And so forth
Dial Setting #2: Moderately-Strict

Suppose a UI developer wants to specify more information than what was originally required for the control.  Consider a push button that performs an unusual action, but might not be intuitive from the push button’s caption or name.  The developer can use the MSAA Description property to describe the action, but it isn’t imperative to the user in operating the control.

Consider an OK button on a dialog. 

  1. The MSAA Default Action property for the button is “Press”.  If this method is not implemented correctly, ie “Click” is returned, or if this method isn’t implemented at all, a failure is reported
  2. The MSAA Value property for the button is “OK”.  A push button is not supposed to implement the Value property.  A failure is reported.
  3. The MSAA Name property for the button is not implemented.  All UI controls must have a name, so this property must be implemented.  A failure is reported.
  4. The MSAA Description property for the button is “Dismisses the dialog”.  A push button isn’t required to support the description property.  But, because the property is implemented, the test suites verify this is the expected description specified by the developer.
  5. And so forth
Dial Setting #3: Moderately-Relaxed

Ignores properties that were implemented, but were not expected to be implemented.   Note in the below example, part ‘b’ does not log a failure.

Consider an OK button on a dialog.

  1. The MSAA Default Action property for the button is “Press”.  If this method is not implemented correctly, ie “Click” is returned, or if this method isn’t implemented at all, a failure is reported.
  2. The MSAA Value property for the button is “OK”.  A push button is not supposed to implement the Value property; however, a failure is not reported, but a warning is logged instead.
  3. The MSAA Name property for the button is not implemented.  All UI controls must have a name, so this property must be implemented.  A failure is reported.
  4. And so forth
Dial Setting #4: Minimum Required level of testing

Perfect for regression tests, all UI controls must support these methods.  These are the highest priority tests that can be run for a control.  From https://msdn.microsoft.com/library/en-us/msaa/msaaccgd_95kk.asp, these properties and methods are Name, Role, State, Parent, Location, Navigate, and HitTest

Consider an OK button on a dialog.

  1. The MSAA Name property for the button is not implemented.  All UI controls must have a name, so this property must be implemented.  A failure is reported.
  2. And so forth
Flowchart describing Sections 1 and 2

Flowchart describing role base testing

Using the above approach in a testing tool or automation framework
Step 1:  Capture the MSAA object for a UI element by either method below
  1. AccessibleObjectFromWindow – the user specifies the window handle (HWND) for the control
  2. AccessibleObjectFromPoint – the (x, y) coordinates of any point within the control’s drawing boundary is specified.  Usually, a tool consists of a pair of crosshairs that the user clicks and drags over the desired control
Step 2:  Pass the generated MSAA object from step 1 into a MSAA Verifier class and the desired level of testing (see Section 2 above)
Step 3:  Obtain the MSAA role for the specified object by calling IAccessible::get_accRole from the MSAA object.
Step 4:  Apply the appropriate test suites for the desired level of testing.  Retrieve corresponding values from win32 API calls (test hooks, etc). 
  1. For example, a push button’s MSAA name comes from its caption.  Retrieve the UI Element’s caption and verify 1. the MSAA name is non-null and 2. it matches the caption from the control. 
  2. Another example, an edit box’s MSAA name comes from the control that immediate precedes it in tab order.  Verify the edit box’s name is the same as the label control’s caption found immediately to the left of the edit box (or above it, if a label isn’t found to the left).
Step 5:  Record test suite results and report