Ensuring that computer programs are accessible to users with disabilities, such as for use with mobile phones

Patent metadata

Field	Value
Publication number	US-11550702-B1
Application number	US-202117519466-A
Country	US
Kind code	B1
Filing date	Nov 4, 2021
Priority date	Nov 4, 2021
Publication date	Jan 10, 2023
Grant date	Jan 10, 2023

Abstract

Disclosed here is a system to enable interaction between a user with a disability and a computer program. The system can obtain a representation of a user interface to present to a user. The system can determine an element associated with the user interface, where the element is configured to provide information to the user, however, the user interface presentation of the element at least partially fails to provide the information to the user. Based on the element, the system can determine an appropriate test to perform. The appropriate test indicates at least two of: a test to perform with a keyboard, a gesture test to perform with a mobile screen reader, and an audio test to perform with a screen reader. The system can generate an indication of the appropriate test. The system can provide the indication of the appropriate test prior to releasing the user interface to the user.

First claim

I claim:

1. A computer-readable storage medium, excluding transitory signals and carrying instructions to generate an indication user interface of an appropriate test to perform to ensure that a user interface provides intended information to a user, which, when executed by at least one data processor of a system, causes the system to:
obtain a representation of the user interface to present to the user;
determine an element associated with the user interface,
wherein the element is configured to provide information to the user,
wherein a user interface presentation of the element at least partially fails to provide the information to the user;
based on the element, determine the appropriate test to perform,
wherein the appropriate test indicates a test to perform with a keyboard, a gesture test to perform with a mobile screen reader, and an audio test to perform with a screen reader;
generate the indication user interface of the appropriate test by:
indicating a keyboard key and an effect caused by activating the keyboard key when the appropriate test includes the test to perform with the keyboard;
indicating a gesture and an effect caused by performing the gesture when the appropriate test includes the gesture test; and
indicating an audio and a function to be performed by the audio when the appropriate test includes the audio test,
wherein the audio test includes a group, and a state,
wherein the group indicates to check whether selecting a visible label associated with the element selects the element, and
wherein the state indicates to check whether the audio correctly describes a selection state associated with the element including selected and unselected; and

provide the indication user interface of the appropriate test prior to releasing the user interface to the user,
wherein prior to releasing the user interface to the user, the appropriate test is performed on the user interface and the user interface passed the appropriate test.

2. The computer-readable storage medium of claim 1 , comprising instructions to:
determine a type of program associated with the user interface, wherein the user interface is a user interface for a mobile phone and wherein the program is a program to run on the mobile phone;
based on the type of program, determine multiple categories associated with the user interface;
categorize the element associated with the user interface into a category among the multiple categories; and
enable a selection of the appropriate test by presenting the multiple categories, the element, and the category associated with the element.

3. The computer-readable storage medium of claim 1 , wherein the user interface is associated with a web page, and comprising instructions to:
determine multiple categories associated with the user interface including at least four of: HTML, header, navigation, main, form, and footer;
categorize the element associated with the user interface into a category among the multiple categories; and
enable a selection of the appropriate test by presenting the multiple categories, the element, and the category associated with the element.

4. The computer-readable storage medium of claim 1 , wherein the user interface is associated with an application, and comprising instructions to:
determine multiple categories associated with the user interface including controls and notifications;
categorize the element associated with the user interface into a category among the multiple categories; and
enable a selection of the appropriate test by presenting the multiple categories, the element, and the category associated with the element.

5. The computer-readable storage medium of claim 1 , wherein the user interface includes a graphical user interface, an audio user interface, or a haptic user interface.

6. The computer-readable storage medium of claim 1 , comprising:
obtain a program associated with the user interface and the indication of the appropriate test,
wherein the appropriate test includes the audio test;

execute the program associated with the user interface and the audio test;
perform natural language processing on the audio test to determine whether the audio test corresponds to an output indicated in the appropriate test; and
upon determining that the audio test does not correspond to the output indicating the appropriate test, indicate that the program did not pass the test.

7. A system comprising:
at least one hardware processor; and
at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to:
obtain a representation of the user interface to present to the user;
determine an element associated with the user interface,
wherein the element is configured to provide information to the user,
wherein a user interface presentation of the element at least partially fails to provide the information to the user;
based on the element, determine the appropriate test to perform,
wherein the appropriate test indicates a test to perform with a keyboard, a gesture test to perform with a mobile screen reader, and an audio test to perform with a screen reader;
generate an indication user interface of the appropriate test by:
indicating a keyboard key and an effect caused by activating the keyboard key when the appropriate test includes the test to perform with the keyboard;
indicating a gesture and an effect caused by performing the gesture when the appropriate test includes the gesture test; and
indicating an audio and a function to be performed by the audio when the appropriate test includes the audio test,
wherein the audio test includes a group, and a state,
wherein the group indicates to check whether selecting a visible label associated with the element selects the element, and
wherein the state indicates to check whether the audio correctly describes a selection state associated with the element including selected and unselected; and

provide the indication user interface of the appropriate test prior to releasing the user interface to the user; and
perform the appropriate test on the user interface;
determine whether the user interface passed the appropriate test; and
upon determining that the user interface passed the appropriate test, indicate that the user interface is ready for release.

8. The system of claim 7 , comprising instructions to:
determine a type of program associated with the user interface;
based on the type of program, determine multiple categories associated with the user interface;
categorize the element associated with the user interface into a category among the multiple categories; and
enable a selection of the appropriate test by presenting the multiple categories, the element, and the category associated with the element.

9. The system of claim 7 , wherein the user interface is associated with a web page, comprising instructions to:
determine multiple categories associated with the user interface including at least four of: HTML, header, navigation, main, form, and footer;
categorize the element associated with the user interface into a category among the multiple categories; and
enable a selection of the appropriate test by presenting the multiple categories, the element, and the category associated with the element.

10. The system of claim 7 , wherein the user interface is associated with an application, comprising instructions to:
determine multiple categories associated with the user interface including controls and notifications;
categorize the element associated with the user interface into a category among the multiple categories; and
enable a selection of the appropriate test by

Assignees

• T Mobile Usa Inc

Inventors

• Triplett Charles Roy

Classifications

• G06F3/0489

  using dedicated keyboard keys or combinations thereof · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G10L15/18

  using natural language modelling · CPC title

• G06F40/14

  Tree-structured documents (parsing G06F40/205; validation G06F40/226) · CPC title