Multimodal text input system, such as for use with touch screens on mobile phones

Patent metadata

Field	Value
Publication number	US-9104312-B2
Application number	US-201113583840-A
Country	US
Kind code	B2
Filing date	Mar 14, 2011
Priority date	Mar 12, 2010
Publication date	Aug 11, 2015
Grant date	Aug 11, 2015

Abstract

A system and method for entering text from a user includes a programmed processor that receives inputs from the user and disambiguates the inputs to present word choices corresponding to the text. In one embodiment, inputs are received in two or more modalities and are analyzed to present the word choices. In another embodiment, a keyboard is divided into zones each of which represents two more input characters. A sequence of zones selected by the user is analyzed to present word choices corresponding to the zone selected.

First claim

We claim:

1. A method, performed using one or more processors and a memory, for data input, the method comprising:
receiving, through an input device and using the one or more processors, a first user input as part of a gesture,
wherein the gesture comprises at least a first input selection that represents less than an entire word and a second input selection; and
wherein the gesture is a continuous stroke across a virtual keyboard from a first simulated key across one or more other simulated keys;

identifying, in the first user input, a bend from the first input selection to a current input location along the gesture;
determining that the bend indicates a third input selection that represents a portion of the word between the first input selection and the second input selection;
assigning to the bend a directional classification from a discrete set of directional classifications;
determining one or more candidates for the third input selection based on the candidates for the third input selection being in a direction, relative to the first input selection, corresponding to the assigned directional classification;
determining, using the one or more processors, one or more possible word suggestions based upon the first input selection and the one or more candidates for the third input selection,
wherein the one or more possible word suggestions are determined prior to receiving the second input selection; and

providing, using a display, the possible word suggestions to the user.

2. The method of claim 1 , wherein the one or more candidates for the third input selection are determined without regard for the proximity of the bend to a representation of the third input selection.

3. The method of claim 1 , wherein the discrete set of directional classifications comprise directional classifications corresponding to up, down, left, and right.

4. The method of claim 1 , wherein the discrete set of directional classifications comprises eight or less directions.

5. The method of claim 1 , wherein assigning to the bend the directional classification comprises:
determining an angle off a horizontal direction or a vertical direction; and
selecting a direction that amounts to one of: up, down, left, or right, wherein selecting the direction is based on a closest correspondence to the determined angle.

6. The method of claim 1 , wherein determining that the bend indicates the third input selection comprises determining that the bend indicates a single character between the first input selection and the second input selection.

7. The method of claim 1 , wherein the first input selection indicates a jumper point for a zone including at least two characters.

8. A non-transitory computer readable medium, storing instructions that, when executed by a computing system, cause the computing system to perform operations for data input, the operations comprising:
receiving a first user input as part of a gesture,
wherein the gesture comprises at least a first input selection that represents less than an entire word and a second input selection; and
wherein the gesture is a continuous stroke across a virtual keyboard from a first simulated key across one or more other simulated keys;

identifying, in the first user input, a bend from the first input selection to a current input location along the gesture;
determining that the bend indicates a third input selection that represents a portion of the word between the first input selection and the second input selection;
assigning to the bend a directional classification from a discrete set of directional classifications;
determining one or more candidates for the third input selection based on the candidates for the third input selection being in a direction, relative to the first input selection, corresponding to the assigned directional classification;
determining one or more possible word suggestions based upon the first input selection and the one or more candidates for the third input selection,
wherein the one or more possible word suggestions are determined prior to receiving the second input selection; and

providing, using a display, the possible word suggestions to the user.

9. The non-transitory computer readable medium of claim 8 , wherein the one or more candidates for the third input selection are determined without regard for the proximity of the bend to a representation of the third input selection.

10. The non-transitory computer readable medium of claim 8 , wherein the discrete set of directional classifications comprise directional classifications corresponding to up, down, left, and right.

11. The non-transitory computer readable medium of claim 8 , wherein the discrete set of directional classifications comprises eight or less directions.

12. The non-transitory computer readable medium of claim 8 , wherein assigning to the bend the directional classification comprises:
determining an angle off a horizontal direction or a vertical direction; and
selecting a direction that amounts to one of: up, down, left, or right, wherein selecting the direction is based on a closest correspondence to the determined angle.

13. The non-transitory computer readable medium of claim 8 , wherein determining that the bend indicates the third input selection comprises determining that the bend indicates a single character between the first input selection and the second input selection.

14. The non-transitory computer readable medium of claim 8 , wherein the first input selection indicates a jumper point for a zone including at least two characters.

15. A computing system configured to perform data input, the system comprising:
one or more processors;
a memory;
a touch sensitive input configured to:
receive, using the one or more processors, a first user input as part of a gesture,
wherein the gesture comprises at least a first input selection that represents less than an entire word and a second input selection; and
wherein the gesture is a continuous stroke across a virtual keyboard from a first simulated key across one or more other simulated keys;

an input recognition module configured to:
identify, in the first user input, a bend from the first input selection to a current input location along the gesture;
determine that the bend indicates a third input selection that represents a portion of the word between the first input selection and the second input selection;
assign to the bend a directional classification from a discrete set of directional classifications;
determine one or more candidates for the third input selection based on the candidates for the third input selection being in a direction, relative to the first input selection, corresponding to the assigned directional classification; and
determine, using the one or more processors, one or more possible word suggestions based upon the first input selection and the one or more candidates for the third input selection,
wherein the one or more possible word suggestions are determined prior to receiving the second input selection; and

a display configured to provide the possible word suggestions to the user.

16. The computing system of claim 15 , wherein the one or more candidates for the third input selection are determined without regard for the proximity of the bend to a representation of the third input selection.

17. The computing system of claim 15 , wherein the discrete set of directional classifications comprise directional classifications corresponding to up, down, left, and right.

18. The computing system of cla

Assignees

• Kay David
• Van Meurs Pim
• Unruh Erland
• Sharan Santosh
• Kessler Gordon
• Agarwal Mayank
• Sigel Greg
• Nuance Communications Inc

Inventors

• Kay David
• Van Meurs Pim
• Unruh Erland
• Sharan Santosh
• Kessler Gordon
• Agarwal Mayank
• Sigel Greg

Classifications

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/04886 Primary

  by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus · CPC title

• G06F40/242

  Dictionaries · CPC title

• G06F3/023 Primary

  Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes · CPC title

• G06F40/274

  Converting codes to words; Guess-ahead of partial word inputs · CPC title