Interface scanning for disabled users

Patent metadata

Field	Value
Publication number	US-11036372-B2
Application number	US-201916716436-A
Country	US
Kind code	B2
Filing date	Dec 16, 2019
Priority date	Jan 25, 2013
Publication date	Jun 15, 2021
Grant date	Jun 15, 2021

Abstract

Systems and processes for scanning a user interface are disclosed. One process can include scanning multiple elements within a user interface by highlighting the elements. The process can further include receiving a selection while one of the elements is highlighted and performing an action on the element that was highlighted when the selection was received. The action can include scanning the contents of the selected element or performing an action associated with the selected element. The process can be used to navigate an array of application icons, a menu of options, a standard desktop or laptop operating system interface, or the like. The process can also be used to perform gestures on a touch-sensitive device or mouse and track pad gestures (e.g., flick, tap, or freehand gestures).

First claim

What is claimed is:

1. A method comprising:
at an electronic device with a display and one or more input devices:
scanning a plurality of elements displayed via the display, wherein scanning the plurality of elements comprises sequentially highlighting the plurality of elements;
receiving, via the one or more input devices, a first selection input while scanning the plural of elements;
in response to receiving the first selection input:
in accordance with a determination that the first selection input is received when a first element of the plurality of elements is highlighted:
displaying, via the display, one or more first sub-elements concurrently with the first element and within a threshold distance of the first element; and
scanning the one or more first sub-elements of the first element;

receiving, via the one or more input devices, a second selection input while scanning the one or more first sub-elements; and
in response to receiving the second selection input:
in accordance with a determination that the second selection input is received when a respective sub-element of the one or more first sub-elements is highlighted, performing an action associated with the respective sub-element of the one or more first sub-elements, wherein the action associated with the respective sub-element comprises:
displaying, via the display, one or more second sub-elements of the respective sub-element, different from the first sub-elements, concurrently with the first element;
ceasing display of the one or more first sub-elements; and scanning the one or more second sub-elements of the respective sub-element.

2. The method of claim 1 , wherein the one or more first sub-elements are displayed in a menu.

3. The method of claim 2 , wherein the one or more first sub-elements displayed in the menu are arranged in two or more rows.

4. The method of claim 2 , wherein the menu overlaps a second element of the plurality of elements without overlapping the first element of the plurality of elements.

5. The method of claim 1 , wherein the one or more second sub-elements of the respective sub-element are displayed in a menu along an edge of the display.

6. The method of claim 1 , wherein the one or more second sub-elements of the respective sub-elements are displayed in a menu that is within the threshold distance of the first element.

7. The method of claim 1 , wherein the one or more second sub-elements comprise one or more gesture sub-elements selectable to perform one or more gestures.

8. The method of claim 7 , wherein the one or more gestures performable by selection of the one or more gesture sub-elements include a tap gesture, a flick gesture, a pan gesture, a pinch gesture, a drag gesture, a tap and hold gesture, a hold and drag gesture, a double tap gesture, or a free-hand gesture.

9. The method of claim 7 , wherein the one or more second sub-elements include a sub-element selectable to adjust a number of fingers used to perform the one or more gestures.

10. The method of claim 7 , further comprising:
displaying, via the display, when scanning the one or more gesture sub-elements, an indicator of a location at which to begin performing the one or more gestures.

11. The method of claim 7 , further comprising:
receiving, via the one or more input devices, a third selection input while scanning the one or more second sub-elements; and
in response to receiving the third selection input:
in accordance with a determination that the third selection input is received when a respective sub-element of the one or more second sub-elements corresponding to a gesture sub-element including a drag operation is highlighted, scanning regions of the display to identify a location at which to end the drag operation.

12. The method of claim 11 , wherein scanning the regions of the display to identify the location at which to end the drag operation comprises:
scanning a first plurality of regions of the display to identify a first coordinate in a first direction in response to a fourth selection input; and
scanning a second plurality of regions of the display to identify a second coordinate in a second direction, different from the first direction, in response to a fifth selection input.

13. The method of claim 12 , further comprising:
after identifying the first coordinate and the second coordinate, displaying, via the display, a visual indicator of the location at which to end the drag operation corresponding to the first coordinate and the second coordinate; and
displaying, via the display, one or more third sub-elements.

14. The method of claim 13 , further comprising:
scanning the one or more third sub-elements;
receiving, via the one or more input devices, a sixth selection input while scanning the one or more third sub-elements; and
in response to receiving the sixth selection input:
in accordance with a determination that the sixth selection input is received when a respective sub-element of the one or more third sub-elements is highlighted, performing an action associated with the respective sub-element of the one or more third sub-elements, wherein the action is associated with the respective sub-element of the one or more second sub-elements.

15. The method of claim 1 , wherein the respective sub-element comprises a gesture element, wherein the one or more second sub-elements comprise one or more gesture sub-elements selectable to perform one or more gestures, and wherein the action associated with respective sub-element further comprises displaying, via the display, an indicator of a location at which to begin performing the one or more gestures.

16. The method of claim 1 , further comprising:
in response to receiving the first selection input:
in accordance with a determination that the first selection input is received when a second element of the plurality of elements, different from the first element of the plurality of elements is highlighted:
displaying, via the display, one or more third sub-elements concurrently with the second element and within a threshold distance of the second element; and
scanning the one or more third sub-elements of the second element;

receiving, via the one or more input devices, a third selection input while scanning the one or more third sub-elements; and
in response to receiving the third selection input:
in accordance with a determination that the third selection input is received when a respective sub-element of the one or more third sub-elements is highlighted, performing an action associated with the respective sub-element of the one or more third sub-elements.

17. The method of claim 1 , wherein the action associated with the respective sub-element further comprises:
scrolling one or more of the plurality of elements displayed via the display;
wherein the one or more second sub-elements of the respective sub-element include one or more sub-elements to control operation of the scrolling.

18. The method of claim 17 , further comprising:
receiving, via the one or more input devices, a third selection input while scanning the one or more second sub-elements and while scrolling the one or more of the plurality of elements; and
in response to receiving the third selection input changing a speed of the scrolling of the one or more of the plurality of elements.

19. An electronic device, comprising:
one or more processors;
a display;
one or more input devices;
memory; and
one or more programs stored in the memory and configured to be executed by the one or mor

Assignees

• Apple Inc

Inventors

• Fleizach Christopher
• Seymour Eric T
• KASEMSET Clare
• White Samuel
• Minifie Darren C
• Fisch Ian
• Hughes Gregory F
• Kaufman Justin
• Hoa Patti P

Classifications

• H04M1/72475

  specially adapted for disabled users · CPC title

• G06F3/04842 Primary

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/0487

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title

• H04M1/2474

  Telephone terminals specially adapted for disabled people (user interfaces for cordless or mobile telephones specially adapted for disabled users H04M1/72475; network based special services or facilities for hearing-impaired persons H04M3/42391; devices for conversing with the deaf-blind G09B21/04) · CPC title