Device, method, and graphical user interface for integrating recognition of handwriting gestures with a screen reader

Patent metadata

Field	Value
Publication number	US-9633191-B2
Application number	US-201414517691-A
Country	US
Kind code	B2
Filing date	Oct 17, 2014
Priority date	Mar 31, 2012
Publication date	Apr 25, 2017
Grant date	Apr 25, 2017

Abstract

While an electronic device with a display and a touch-sensitive surface is in a screen reader accessibility mode, the device displays an application launcher screen including a plurality of application icons. A respective application icon corresponds to a respective application stored in the device. The device detects a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters. A respective gesture that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character. The device determines whether the detected sequence of one or more gestures corresponds to a respective application icon of the plurality of application icons, and, in response to determining that the detected sequence of one or more gestures corresponds to the respective application icon, performs a predefined operation associated with the respective application icon.

First claim

What is claimed is:

1. A method, comprising:
while an electronic device with a display and a touch-sensitive surface is in a screen reader accessibility mode:
displaying at least a portion of a web page on the display, the web page including a plurality of user interface elements;
detecting a first navigation gesture on the touch-sensitive surface;
in response to detecting the first navigation gesture on the touch-sensitive surface, navigating to a first set of one or more user interface elements of the plurality of user interface elements that corresponds to a current navigable unit type, wherein the current navigable unit type is set to a first navigable unit type selected from a plurality of predefined navigable unit types, wherein the plurality of predefined navigable unit types includes at least four different navigable unit types including the first navigable unit type, a second navigable unit type, a third navigable unit type, and a fourth navigable unit type;
detecting a navigation setting gesture on the touch-sensitive surface that corresponds to a respective character, wherein:
the navigation setting gesture that corresponds to the respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character;

determining whether the navigation setting gesture corresponds to one of the second navigable unit type, the third navigable unit type, and the fourth navigable unit type;
in accordance with a determination that the navigation setting gesture corresponds to the second navigable unit type of the plurality of predefined navigable unit types:
changing the current navigable unit type from the first navigable unit type to the second navigable unit type; and
outputting audible accessibility information indicating that the second navigable unit type has been selected;

in accordance with a determination that the navigation setting gesture corresponds to the third navigable unit type of the plurality of predefined navigable unit types:
changing the current navigable unit type from the first navigable unit type to the third navigable unit type; and
outputting audible accessibility information indicating that the third navigable unit type has been selected;

in accordance with a determination that the navigation setting gesture corresponds to the fourth navigable unit type of the plurality of predefined navigable unit types:
changing the current navigable unit type from the first navigable unit type to the fourth navigable unit type; and
outputting audible accessibility information indicating that the fourth navigable unit type has been selected;

after changing the current navigable unit type, detecting a second navigation gesture on the touch-sensitive surface; and,
in response to detecting the second navigation gesture on the touch-sensitive surface, navigating to a second set of one or more user interface elements of the plurality of user interface elements that corresponds to the changed current navigable unit type.

2. The method of claim 1 , wherein:
the respective path traverses one or more locations on the touch-sensitive surface that corresponds to one or more user interface elements of the plurality of user interface elements in the web page without activating the one or more user interface elements.

3. The method of claim 1 , wherein the respective path is in the shape of the respective character.

4. The method of claim 1 , wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to any of the plurality of user interface elements.

5. The method of claim 1 , wherein:
the plurality of predefined navigable unit types includes four or more navigable unit types in a circular sequence so that a navigable unit type that has a second position in the circular sequence is deemed to be adjacent to a navigable unit type that has a first position in the circular sequence, the navigable unit type that has a third position in the circular sequence is deemed to be adjacent to the navigable unit type that has the second position in the circular sequence, and a navigable unit type that has a last position in the circular sequence is deemed to be adjacent to the navigable unit type that has the first position in the circular sequence; and
the first navigable unit type is not adjacent to the second navigable unit type in the circular sequence.

6. The method of claim 5 , including:
detecting a second navigation setting gesture on the touch-sensitive surface, wherein the second navigation setting gesture is a multi-finger gesture, and the second navigation setting gesture is independent of contacting a location on the touch-sensitive surface that corresponds to any of the plurality of user interface elements; and,
in response to detecting the second navigation setting gesture on the touch-sensitive surface:
changing the current navigable unit type from the second navigable unit type to a third navigable unit type that is adjacent to the second navigable unit type in the sequence of the plurality of navigable unit types; and
outputting accessibility information about the third navigable unit type.

7. The method of claim 1 , including:
while the electronic device is in the screen reader accessibility mode, prior to detecting the navigation setting gesture on the touch-sensitive surface:
detecting a predefined input; and,
in response to detecting the predefined input, entering a handwriting gesture recognition mode within the screen reader accessibility mode.

8. The method of claim 1 , wherein a label for the second navigable unit type starts with the respective character.

9. An electronic device, comprising:
a display;
a touch-sensitive surface;
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
while the electronic device is in a screen reader accessibility mode:
displaying at least a portion of a web page on the display, the web page including a plurality of user interface elements;
detecting a first navigation gesture on the touch-sensitive surface;
in response to detecting the first navigation gesture on the touch-sensitive surface, navigating to a first set of one or more user interface elements of the plurality of user interface elements that corresponds to a current navigable unit type, wherein the current navigable unit type is set to a first navigable unit type selected from a plurality of predefined navigable unit types, wherein the plurality of predefined navigable unit types includes at least four different navigable unit types including the first navigable unit type, a second navigable unit type, a third navigable unit type, and a fourth navigable unit type;
detecting a navigation setting gesture on the touch-sensitive surface that corresponds to a respective character, wherein:
the navigation setting gesture that corresponds to the respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character;

determining whether the navigation setting gesture corresponds to one of the second navigable unit type, the third navigable unit type, and the fourth navigable unit type;
in accordance with a determination that the navigation setting gesture corresponds to the second navigable unit type of the plurality of predefined navigable unit types:
changing the current navigable unit type from the first navigable unit type to the second navigable unit type; and
outputting au

Assignees

• Apple Inc

Inventors

• Fleizach Christopher B
• Minifie Darren C
• Hughes Gregory F
• Dour Ryan N
• Fisch Ian M
• Lopes Da Silva Joel M
• Pedersen Ii Michael M
• Seymour Eric T
• Naik Devang K
• Dixon Ryan S

Classifications

• G06F21/36 Primary

  by graphic or iconic representation · CPC title

• G06F3/04883 Primary

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/04817

  using icons (graphical or visual programming using iconic symbols G06F8/34) · CPC title

• G06F3/04886

  by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title