Disadvantages or Testing with Screen Readers Alone

Submitted by rmusachio on Thu, 2009-05-14 16:50.

Introduction
A misconception exists that if an electronic and information technology (E&IT) product is deemed accessible by a web content developer who uses a screen reader as an analysis tool, the product must be completely accessible. This is not the case. Even though a software or web-based application is tested with a screen reader, the product still may have issues

This brief will examine the pitfalls of conducting accessibility evaluations with only a screen reader. The discussion will be based on components that are tested against Section 508 Software Applications and Operation Systems standards (1194.21), Web-based Information and Applications standards (1194.22), and Worldwide Web Consortium Web Accessibility Guidelines (WCAG).

Graphics and Images
Screen readers can detect missing Alt and longdesc attributes from web-based graphics, lacking text labels from interface elements in software programs, and absent captions from multimedia presentations. Yet, the AT device alone cannot determine if the text equivalent accurately represents the corresponding element. For instance, if a webpage has a complex graph with a longdesc attribute that has inaccurate details, the screen reader cannot capture the inaccuracy. Since the graph has a text equivalent, sighted analysts automatically assumes that it is accessible unless they really consider its content.

Additionally, screen readers may be unable to detect if bitmap images are constant with interface controls in software applications. Since control functions rather than the images are labeled, a screen reader could not know if an image of a pencil consistently represents Address Book functions throughout a software program. Once again, a visual evaluation is required.

Color
Screen readers also cannot be used as an evaluation tool to detect color conveyed information unless it is accompanied by a text equivalent, as in the following example:

The first three names are finalists in the contest:
Jody
Tracy
Brad

Brittney
Tim
Greg
Pam

If the identifying sentence were not above the names, a screen reader could not give any indication that the names in red are the finalists.

Screen readers further cannot help evaluate color contrast issues. As with color conveyed information, screen readers cannot identify background and foreground colors. A more appropriate tool to detect color issues would be the Accessibility Toolbar. The toolbar renders a page in black and white, list all the colors used on the page, and performs color contrast analysis.

Keyboard Access
Although keyboard accessibility of an E&IT product can be evaluated by using a screen reader, results may be skewed due to the AT device’s capabilities. To illustrate, sometimes the complex shortcuts of JAWS screen readers activate elements that cannot be activated by using the keyboard only. An analyst using a screen reader may be able to activate a graphic button by pressing Insert E. Conversely, if the tester were to use the keyboard only, the button may not be reachable with the TAB key. Therefore, a screen reader and then a keyboard alone should be used as evaluation tools in this circumstance.

Focus
When an element obtains focus, screen readers indicate it by announcing its text equivalent, such as “Submit” on a button. However, screen readers cannot indicate if the visual focus is dark enough or if it even exists. Furthermore, the AT device cannot determine if the visual focus follows a logical tab order.

Blinking and Flashing
Another issue that screen readers may have difficulty detecting accurately is blinking or flashing objects. Of course, the analyst can check the HTML code if flickering exists by using a screen reader, but it can be done with the AT device much more easily.

Tables
Through the HTML source code, a tester can use a screen reader to determine if a data table is accessible up to a point. The analyst can test if a data table’s column or row heading is announced with its corresponding data cell. Yet, using a screen reader does not enable the analyst to check if columns and rows lack text headings, especially if a header cell is marked up with TD instead of TH tags.

Forms
Screen readers certainly can inform analysts if form fields have explicit labeling. They even can tell whether implicitly associated labels (visual text labels) are positioned properly to their corresponding form fields. However, difficulties can arise if a form field launches an error message in a dialog window. The screen reader tester would have to return to the initial window to locate the erroneous form field.

Another issue regarding form fields is that screen readers cannot indicate how form elements should be grouped. As the example below shows, two edit blanks labeled “Name” exist on an ordering webpage. If the page lacks markup, an analyst using a screen reader may not recognize that the first edit blank is for the “Shipping” name while the second edit blank is for the “Billing” name since the AT device reads content horizontally from left to right. Although the screen reader would alert the tester that an issue exists since it would read “name” twice, he still would have to examine the page visually to determine which “Name” field goes in which group.

Shipping Billing

Name __________ Name_________

Scripts
Scripts, like Javascript, often pose obstacles for screen reader users. Experienced users can detect if a scripted element has a text equivalent or if it can be activated through JAWS shortcuts. However, content developers who evaluate their scripted work by using screen readers may not know all the shortcuts and other functions of the AT devices.

Conclusion
As mentioned above, experience levels of screen reader testers also can affect accessibility testing. For instance, the native AT user with a severe vision impairment may be able to determine how scripts function on a webpage. Conversely, analyzers who have never used screen readers before may not know how to access a website with the AT devices.

When testing a website or software application, content developers should not focus only on the issues that affect persons with visual impairments who use screen readers but also on obstacles that face individuals with other disabilities. Testers with various disabilities can discover issues that affect accessibility towards users with the same impairments. An analyst with color blindness, for example, concludes that the font color is too light for a webpage background, or a tester who uses a headpointer experiences form fields that are not reachable with the TAB key. The former finding would help remediate accessibility for other users with color blindness while the latter result would help other persons with dexterity disabilities access the form elements.

Furthermore, testers with vision impairments who are skilled at using screen readers must be a viable part of an accessibility evaluation plan. Otherwise, issues such as missing heading tags or inappropriate data table markup might be overlooked. Results obtained by non-visually impaired testers, however, may facilitate accessibility towards screen reader users. For instance, a sighted tester may notice that an image does not match its text equivalent.

To ensure Section 508 compliancy and accessibility, therefore, web developers and content analysts should not perform evaluations based on screen reader usage only. Instead, they must include individuals with diverse disabilities and AT capabilities in the testing plan.