Neural network for keyboard input decoding

Patent metadata

Field	Value
Publication number	US-10248313-B2
Application number	US-201715473010-A
Country	US
Kind code	B2
Filing date	Mar 29, 2017
Priority date	Apr 10, 2015
Publication date	Apr 2, 2019
Grant date	Apr 2, 2019

Abstract

In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.

First claim

What is claimed is:

1. A method comprising:
outputting, by a computing device, for display, a graphical keyboard that includes a plurality of keys;
receiving, by the computing device, a gesture that selects one or more keys from the plurality of keys;
determining, by the computing device, based on the gesture and a Long Short Term Memory network, at least one character string; and
outputting, by the computing device, for display, the at least one character string.

2. The method of claim 1 , wherein determining the at least one character string comprises:
determining, based on the gesture, one or more input values to the Long Short Term Memory network, the one or more input values corresponding to one or more features; and
determining, based on the one or more input values, one or more output values from the Long Short Term Memory network, the one or more output values corresponding to the one or more features; and
determining, based on the one or more output values, the at least one character string.

3. The method of claim 2 , wherein the one or more features comprise at least one of spatial features, temporal features, lexical features, and contextual features.

4. The method of claim 3 , wherein the spatial features include one or more of: a location of the gesture relative to a presence-sensitive input device, a speed of the gesture, a direction of the gesture, a curvature of the gesture, a particular key from the plurality of keys that is traversed by the gesture, or a type of the gesture.

5. The method of claim 3 , wherein the temporal features include information about a time at which the gesture is received.

6. The method of claim 3 , wherein the lexical features include one or more of one or more characters, character strings, or multi-character string phrases associated with other gestures received prior to the gesture.

7. The method of claim 3 , wherein the contextual features include one or more of an identity of a user of the computing device, a geolocation of the computing device, an environment or climate of the computing device, audio or visual information determined by the computing device, sensor information detected by the computing device, a type of input field associated with text determined from the gesture, an application associated with the text determined from the gesture, or a recipient associated with the text determined from the gesture.

8. The method of claim 2 , wherein the one or more output values that correspond to one or more features comprise a plurality of one or more of characters, character strings, or multi-string phrases and a respective probability for each of the plurality of one or more of characters, character strings, or multi-string phrases.

9. The method of claim 1 , wherein outputting the at least one character string for display comprises outputting the at least one character string for display in a word suggestion region of the graphical keyboard.

10. The method of claim 1 , wherein the Long Short Term Memory network comprises at least one of: one or more memory block or a plurality of layers of memory blocks.

11. The method of claim 1 , wherein the Long Short Term Memory network is configured to execute at one or more local processors of the computing device.

12. The method of claim 1 , wherein the computing device does not include an n-gram language model.

13. The method of claim 1 , wherein the computing device does not include an n-gram spatial model.

14. The method of claim 1 , wherein the gesture is at least one of a tap gesture, continuous gesture, or combination of tap gesture and continuous gesture.

15. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause at least one processor to:
output, for display, a graphical keyboard that includes a plurality of keys;
receive a gesture that selects one or more keys from the plurality of keys;
determine, based on the gesture and a Long Short Term Memory network, at least one character string; and
output, for display, the at least one character string.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed, further cause the at least one processor determine the at least one character string by at least:
determining, based on the gesture, one or more input values to the Long Short Term Memory network, the one or more input values corresponding to one or more features; and
determining, based on the one or more input values, one or more output values from the Long Short Term Memory network, the one or more output values corresponding to the one or more features; and
determining, based on the one or more output values, the at least one character string.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed, further cause the at least one processor to output the at least one character string for display by at least outputting the at least one character string for display in a word suggestion region of the graphical keyboard.

18. A computing device comprising:
at least one processor configured to:
output, for display, a graphical keyboard that includes a plurality of keys;
receive a gesture that selects one or more keys from the plurality of keys;
determine, based on the gesture and a Long Short Term Memory network, at least one character string; and
output, for display, the at least one character string.

19. The computing device of claim 18 , wherein the at least one processor is configured to determine the at least one character string by at least:
determining, based on the gesture, one or more input values to the Long Short Term Memory network, the one or more input values corresponding to one or more features; and
determining, based on the one or more input values, one or more output values from the Long Short Term Memory network, the one or more output values corresponding to the one or more features; and
determining, based on the one or more output values, the at least one character string.

20. The computing device of claim 18 , wherein the at least one processor is configured to output the at least one character string for display by at least outputting the at least one character string for display in a word suggestion region of the graphical keyboard.

Assignees

• Google Inc
• Google Llc

Inventors

• Zhai Shumin
• Breuel Thomas
• Alsharif Ouais
• Ouyang Yu
• Beaufays Francoise
• Schalkwyk Johan

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06F40/279

  Recognition of textual entities · CPC title

• G06F40/232

  Orthographic correction, e.g. spell checking or vowelisation · CPC title

• G06F3/0237

  using prediction or retrieval techniques · CPC title

• G06F3/0219

  Special purpose keyboards · CPC title