How To Really Do Screen Reader Testing (or Assistive Technology Testing)

  • Article

In the past, I’ve focused on testing MSAA Properties. Now we’re ready to take screen reading testing to the next level by adding Events into the algorithm. It is not only essential that a control support the correct MSAA properties, but it is also essential that the control fire the right event for these properties.

Here is my new and approved Screen Reader Testing Model.

Perform initial sanity checking using either AccExplorer or Inspect to view the contents of the Accessible Object.

Step 1: Verify the control contain an AA object. A nameless window container is not a valid AA object for controls like List views, buttons, edit boxes, static text, and so forth.

Step 2: Verify the AA object has the correct role. For example, a list view is not a tree view, so it is not acceptable for a tree view control to be represented by a list view in the AA model.

Step 3: Verify tab order is correct. Point AccExplorer at the entire dialog. Then arrow down through the children. Make sure you have the yellow rectangle highlight selected, so that you can follow visually which objects are now in focus. The tab order must match the way the controls area laid out on the dialog, namely they should start in upper left corner and make their way down to the bottom right. AccExplorer lists the children of the dialog in tab order. To be more precise, the child IDs of the AA objects from the dialog are listed in tab order, so that’s how you can use that effect to test the dialog.

So much of MSAA is dependent on tab order. For example, an edit box gets its name from the label that immediately precedes it in tab order. So, if the tab order is incorrect, the edit box may not have an AA name, or even worse, have an AA name (or label) for another edit box.

Step 4: Verify focus never disappears. Focus must never ever disappear. Do all controls that require tab stops have tab stops? Ensure that no interaction is required via mouse alone.

Step 5: Verify all controls have AA names. All controls must have a name, except for a few exceptions, like a list view when it doesn’t have a label because of the design. Look at Windows Explorer. However, I don’t believe that this is necessarily correct. You can always put an invisible label in front of the list view to give it an AA name. The sighted user will never see the label (unless running Spy++ or AccExplorer), but the screen reader user will be able to read the list view’s AA name.

Perform deeper AA testing using either AccExplorer or Inspect to view the contents of the Accessible Object.

Step 6: Verify all controls have their correct MSAA Properties. Please see my post Which Controls Support Which MSAA Properties, and How These Controls Implement MSAA Properties.

Use AccEvent to verify the correct events are fired and your application does not crash with Assistive Technologies running

Step 7: Verify your application does not crash by running AccEvent while running user-scenarios on your application

First, you’ll need to enable all options for AccEvent.

  1. Launch AccEvent
  2. Select All Object Properties
  3. Select All Event Information
  4. Hit Apply / OK

Now with AccEvent running, interact with each control either through mouse or keyboard. You want to get the control to fire at least 1 event per event type. You’ll also want to perform as many user scenarios as possible. If the application crashes with AccEvent running, there exists a very high probability that the application will crash with an assistive technology (or screen reader) running. Also, if running accEvent causes a crash, it is very highly likely the application caused the crash.

If you have automated tests, try running all of your automated tests that interact with the UI while AccEvent is running minimized. Aren’t automated tests great! <grins>

Step 8: Verify that your MSAA implementation fires the correct event

The MSAA SDK has a listing of all possible Accessibility Events, along with detailed descriptions, located at https://msdn.microsoft.com/library/en-us/msaa/msaaccrf_7jlf.asp

Events that apply to all objects

EVENT_OBJECT_SHOW

A hidden object is shown. If an object contains children, only the parent object fires the event. For example, suppose a list view appears, that was previously invisible, with 5 list items. Only one event is fired.

An EVENT_OBJECT_SHOW event indicates that the STATE_SYSTEM_INVISIBLE bit is no longer set. It is overkill to fire an EVENT_STATE_CHANGE event for the invisible bit in this case.

EVENT_OBJECT_HIDE

An object is hidden. If an object contains children, only the parent object fires the event.

The EVENT_OBJECT_HIDE event indicates that the STATE_SYSTEM_INVISIBLE flag has been set. It is overkill to fire an EVENT_STATE_CHANGE event for the invisible bit in this case.

EVENT_OBJECT_REORDER

An object has added, removed, or reordered its children. For example, this event is generated by a list view object when the number of child elements or the order of the elements changes.

These events only apply to objects that support these states.

EVENT_OBJECT_FOCUS

An object has received the keyboard focus.

EVENT_OBJECT_SELECTION

The selection within an object that has children has changed.

EVENT_OBJECT_STATECHANGE

An object’s state has changed. For example, a state change occurs when a button object is pressed or released, or when an object is enabled or disabled.

EVENT_OBJECT_LOCATIONCHANGE

An object has changed location, shape, or size.

EVENT_OBJECT_VALUECHANGE

An object’s value property has changed.

Examples for testing using AccEvents

Example 1: How to Test for State Change Event (EVENT_STATE_CHANGE)

  1. Open Control Panel – Accessibility Options
  2. Check the “Use StickyKeys” checkbox
  3. Point AccExplorer at the Use StickyKeys checkbox
  4. Verify that the AA state shows “checked.” If this state isn’t set, the event fired will not contain the correct information.
  5. Uncheck “Use StickyKeys”
  6. Verify that the AA state shows normal
  7. Launch AccEvent with only OBJ_STATECHANGE selected from Events. This can be found under Options – Settings.
  8. Check the “Use StickyKeys” checkbox (either mouse or keyboard will work)
  9. Verify that checked appears for “Use StickyKeys” checkbox’s State Change event.

OBJ_STATECHANGE Name="Use StickyKeys" Role=check box State=focused,checked,focusable

If checked doesn’t appear, re-verify step 4. If OBJ_STATECHANGE isn’t fired, then it is a bug against the checkbox.

Example 2: How To Test for Focus Change Event (EVENT_FOCUS_CHANGE)

  1. Open Control Panel – Accessibility Options
  2. Verify that all controls that can receive user input receive focus via the keyboard
  3. Launch AccEvent with only OBJ_FOCUSCHANGE selected from Events. This can be found under Options – Settings.
  4. Tab to each control
  5. Verify that a focus changed event is fired for each object. For example, when tabbing from “Use StickyKeys” to “Settings”, the following info is recorded:

OBJ_FOCUS Name="Settings" Role=push button State=focused,focusable

Other Examples To Pay Attention To

Verify all tree view nodes fire state change events whenever you expand or collapse them.

Verify selection change events are fired when navigating among items in a list.

Verify a state change event is fired for all check boxes.

Verify if an action causes a new control to appear, the corresponding Show event is fired for that object.

Troubleshooting Screen Reading Issues where issues are against the application

Does the screen reader read only a few properties or does the screen reader read nothing at all?

If the screen reader reads only a few properties, which properties are not being read? Are they set in MSAA? If they are not set in MSAA, it is most likely a bug against the application. If they are set correctly in MSAA, are the correct AccEvents being fired? If no events are being fired, it is most likely a bug against the application.

If the screen reader doesn’t read at all, is an EVENT_FOCUS_CHANGE event fired? If yes, are the properties in the event meaningful in accordance with Which Controls Support Which MSAA Properties, and How These Controls Implement MSAA Properties? If the properties are not meaningful or no events are being fired, it is most likely a bug against the application.