UI Automation support in the NVDA Screen Reader

  • Article

NV Access has been able to implement support in NVDA for Microsoft's unmanaged UI Automation client API found in the up and coming Windows 7 Operating System. NV Access developers have been working on this support since the start of this year and just a few days ago decided that it was complete enough to be included in the main development branch for NVDA, which means that this support is now available to the public via the daily NVDA snapshots.

NV Access is asking that any NVDA users that have access to the Windows 7 Release Candidate please try out an NVDA snapshot on Windows 7, and provide feedback, so that they can continue to improve and tidy up support for Windows 7, before both the next version of NVDA, and the Windows 7 Operating System, are released officially.

From a user point of view, there are a few specific situations where having UI Automation support in NVDA improves usability in Windows 7. Such as:

  • The search field in Windows Explorer and the Start Menu. As this search control is no longer a standard Edit field like in previous Windows Operating Systems, it is necessary for NVDA to make use of the UI Automation text pattern in order to track the caret and access the text content.

 

  • Announcement of row and column info in the new Windows Explorer Items View. Unlike previous Windows Operating Systems, Windows 7's Windows Explorer uses a new Items View control to present the files and folders (replaces the older ListView? control). Although this control can be accessed pretty well just using MSAA, using UI Automation has the advantage of providing row and column information for any given item. This is not only useful when navigating multiple columns of files, but this is also useful if you choose to view your files in Details View, and you use the left and right arrow keys to navigate the fields of the item (date modified, size, type etc). This field navigation is a new feature that the Items View provides.
 
  • Reading basic XPS documents in XPS Viewer. XPS Viewer is an application that is included in Windows 7 which allows you to view XPS documents. XPS documents are yet another file format which tries to maintain actual presentation across systems and devices. To allow screen readers and other assistive technologies access to the document content, XPS viewer implements some support for UI Automation. NVDA of course makes use of this where it can, thus allowing users to navigate documents, using the NVDA review cursor. For now a more regular arrows-based mode of navigation in XPS Viewer is not possible as XPS Viewer does not allow you to move its caret (editing cursor), and we have not yet considered implementing any kind of fake cursering such as a cursor manager or virtual buffer. Note also that due to a bug in XPS Viewer, navigating by word or line navigates by paragraph. As this is definitly not suitable, we have made sure Microsoft is aware of this problem.

Like support in NVDA for other APIs such as IAccessible2, NV Access' support for UI Automation is not locked down to any specific control or application. This means that if any application in Windows 7, or even a third party application correctly implements UI Automation natively, NVDA will make use of it. When using NVDA with Windows 7, it is most likely that you will come in to contact with UI Automation in many more situations than the ones listed above. However, as UI Automation is rather new in the Windows Operating System, it is likely that most support you will get will closely mirror what you would get with pure MSAA (I.E. using an older non-UIAutomation version of NVDA will probably provide the same access). When running in Windows 7, NVDA still must in many situations make use of either MSAA, or other native win32 APIs in order to provide suitable access to the Operating System. We definitly do look forward to future Windows releases where UI Automation may be implemented more fully through out the Operating System more widely, providing one simple way of gaining access to all controls, including standard Edit and RichEdit fields.