Neural network for keyboard input decoding

Patent metadata

Field	Value
Publication number	US-9678664-B2
Application number	US-201514683861-A
Country	US
Kind code	B2
Filing date	Apr 10, 2015
Priority date	Apr 10, 2015
Publication date	Jun 13, 2017
Grant date	Jun 13, 2017

Abstract

In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.

First claim

What is claimed is:

1. A computing device comprising:
at least one processor; and
at least one module, operable by the at least one processor to:
output, for display at an output device operatively coupled to the computing device, a graphical keyboard;
receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard;
determine, based on a neural network processing at least one spatial feature of the gesture, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture and the neural network comprises a Long Short Term Memory network; and
output, for display at the output device, the at least one character string determined based on the neural network processing of the at least one spatial feature of the gesture using the neural network.

2. The computing device of claim 1 , wherein the at least one module is operable by the at least one processor to determine the at least one character string by:
inputting, into the neural network, a plurality of input values for a plurality of features, wherein the plurality of features includes the at least one spatial feature;
applying one or more functions of the neural network to the plurality of input values; and
determining, based at least in part on the neural network, one or more output values; and
determining, based at least in part on at least one of the one or more output values, the at least one character string.

3. The computing device of claim 2 , wherein:
the one or more output values are one or more first output values,
the plurality of input values is a plurality of first input values, and
the at least one module is further operable by the at least one processor to determine the at least one character string by:
storing, in the neural network, state information that is based at least in part on the one or more first output values;
inputting, into the neural network, a plurality of second input values for the plurality of features, wherein the plurality of second input values are determined after the plurality of first input values;
applying the one or more functions of the neural network to the plurality of second input values;
determining, based at least in part on the state information, one or more second output values; and
determining, based at least in part on at least one of the one or more second output values, the at least one character string.

4. The computing device of claim 2 , wherein:
the plurality of input values is included in cells of an input matrix,
each respective column vector of the input matrix comprises a respective set of input values of the plurality of input values,
the respective set of input values corresponds to at least one of a particular gesture or portion of a particular gesture,
the plurality of output values is included in cells of an output matrix,
each respective column vector of the output matrix comprises a respective set of output values of the plurality of output values, and
the respective set of output values indicates one or more probabilities of at least one or more characters or one or more character strings.

5. The computing device of claim 1 , wherein the Long Short Term Memory network includes at least one memory block.

6. The computing device of claim 5 , wherein the Long Short Term Memory network includes a plurality of layers of memory blocks.

7. The computing device of claim 1 , wherein the at least one module is operable by the at least one processor to determine the at least one character string by:
performing, based on the neural network processing the at least one spatial feature of the gesture, at least one of auto-prediction, auto-correction, or auto-completion to generate the at least one character string.

8. The computing device of claim 1 , wherein the at least one module is operable by the at least one processor to:
train, based at least in part on a training set, the neural network prior to receiving the indication of the gesture detected at the location of the presence-sensitive input device and prior to determining the at least one character string.

9. The computing device of claim 1 , wherein the computing device does not include a language model and a spatial model.

10. The computing device of claim 1 , wherein the gesture is at least one of a tap gesture, continuous gesture, or combination of tap gesture and continuous gesture.

11. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause at least one processor to:
output, for display at an output device operatively coupled to the computing device, a graphical keyboard;
receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard;
determine, based on at least feature that is processed by the computing device using a neural network, at least one character string, wherein the neural network comprises a Long Short Term Memory network; and
output, for display at the output device, based at least in part on the processing of the at least feature that is processed by the computing device using the neural network, the at least one character string.

12. The non-transitory computer readable storage medium of claim 11 , wherein the instructions, when executed, cause the at least one processor to determine the at least one character string by:
inputting, into the neural network, a plurality of input values for a plurality of features, wherein the plurality of features includes the at least one spatial feature;
applying one or more functions of the neural network to the plurality of input values;
determining, based at least in part on the neural network, one or more output values; and
determining, based at least in part on at least one of the one or more output values, the at least one character string.

13. The non-transitory computer readable storage medium of claim 12 , wherein:
the one or more output values are one or more first output values,
the plurality of input values is a plurality of first input values, and
the instructions, when executed, further cause the at least one processor to determine the at least one character string by:
storing, in the neural network, state information that is based at least in part on the one or more first output values;
inputting, into the neural network, a plurality of second input values for the plurality of features, wherein the plurality of second input values are determined after the plurality of first input values;
applying the one or more functions of the neural network to the plurality of second input values;
determining, based at least in part on the state information, one or more second output values; and
determining, based at least in part on at least one of the one or more second output values, the at least one character string.

14. The non-transitory computer readable storage medium of claim 11 , wherein the Long Short Term Memory network includes at least one memory block.

15. The non-transitory computer readable storage medium of claim 11 , wherein the Long Short Term Memory network includes a plurality of memory blocks.

16. A method comprising:
outputting, by a computing device and for display at an output device operatively coupled to the computing device, a graphical

Assignees

• Google Inc

Inventors

• Zhai Shumin
• Breuel Thomas
• Alsharif Ouais
• Ouyang Yu
• Beaufays Francoise
• Schalkwyk Johan

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06F3/04895 Primary

  Guidance during keyboard input operation, e.g. prompting · CPC title

• G06F40/232

  Orthographic correction, e.g. spell checking or vowelisation · CPC title

• G06F3/0233 Primary

  Character input methods · CPC title

• G06F3/04886 Primary

  by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus · CPC title