CodeTalk: Making IDEs more accessible for developers with visual impairment

The graphically dense interface of integrated development environments (IDEs) makes them immensely powerful for developers across the world. Visual cues are used extensively in these environments to help developers get more information about their code with little more than a quick glance. However, these visual cues are inaccessible to many developers with visual impairments.

According to the World Health Organization, there are 285 million people who cannot read all the content on a regular screen due to some form of visual impairment. Of these 39 million people are blind and cannot access any visual information on screens.

Bringing technology and cutting-edge tools to people with disabilities is part of our mission to empower everyone, everywhere. CodeTalk helps developers with visual impairment to get involved in creating and developing new technologies with the power of code. We are trying to make development platforms more accessible to everyone.

Improving the experience and productivity for developers with visual impairment

IDEs are designed to boost productivity. Developers on the platform have access to a range of tools that help them monitor the code, rectify errors, debug, and modify their programs. Bright colors indicate code syntax, squiggly red lines underscore errors in the code, and multiple windows are required to run a debugging process. Graphical models are used to represent the code structure, the performance of the programme and bottlenecks in the architecture.

Although developers with low vision and severe visual impairments can use IDEs with the aid of screen readers, the experience is incomplete since screen readers miss out on these essential visual cues. A screen reader, for example, cannot describe a bar chart or graphical model on the platform. This is a problem even for coders with visual impairment who use a refreshable braille display. Not only does this impede productivity, but it also makes the experience of creating something on the platform needlessly frustrating.

Project CodeTalk

Project CodeTalk is our ongoing effort to address these issues. By rethinking the design of our IDEs, defining accessibility guidelines for IDEs, we can make powerful development platforms more accessible to developers.

For example, extensions developed for IDEs, such as Visual Studio and VS Code remove three key barriers to usage:

i) Discoverability

Converting graphical user interfaces (GUIs) into audio user interfaces (AUIs) helps developers with visual impairment use platforms at par with everyone else. Audio cues and specific non-verbal sounds can alert the user about bugs, errors and syntax issues. Users can then access a list of errors with a single keystroke. Audio bridges the accessibility gap and allows users to discover more information on their code without the need for visual cues.

Talk Point

Debugging the code is one of the crucial aspects of IDEs that are made accessible with ‘Talk Points’. Talk Points can announce the result of an expression when a certain line of code is reached and help the developer choose whether to continue or stop the execution of the program. Talk Points can be based on speech, where specific statements are spoken when a Talk Point is hit. They can also be programmed to play a tone to let users know whether a certain block has been executed or not. Users with visual impairment can also modify the Talk Points to read out expressions in the execution context so that developers can get contextual information without changing the underlying code.

ii) Glanceability

Tools built into the CodeTalk extension can help users get quick information about the code structure without the need for visual representations. The file structure, code summaries, and function lists can be described through the screen reader to help developers get a holistic view of the project. Code Summary allows developers to directly navigate through the code construct with a handy summary of the construct in an accessible window. Function Lists provide a list of active functions that are easily navigable. These tools can also help users detect the current location of the cursor through a list of blocks, functions, classes and namespaces in the code structure.

Code Summary

Function Lists

iii) Navigability

CodeTalk allows users to move to context and place the cursor at specific points in the code structure with a simple keystroke. For example, a keystroke can help the user bring the cursor to the start or end point of a block of code. Similarly, keystrokes can help users locate and skip comments on their document.

These enhanced features, such as audio cues and additional feedback on their code have received positive feedback from a group of developers with visual impairment.

Enabling the shared goal of wider accessibility

Accessibility is centred around end-user needs. At Microsoft, our commitment to accessibility extends to our entire product spectrum as we endeavour to deliver great experiences to people with disabilities – be they end-users or developers. By incorporating accessibility considerations in the development environment, CodeTalk aims to systematically address barriers for developers with visual impairment. We’re committed to investing more and more efforts in this evolving medium to enable the shared goal of wider accessibility – not just for the users of software but the creators themselves.

For more information on CodeTalk and its features or to download the latest VISX installer, please visit: https://microsoft.github.io/CodeTalk/

With contributions from Suresh Parthasarathy, Gopal Srinivasa and Priyan Vaithilingam.