Combined speech and touch input for observation symbol mappings

Patent metadata

Field	Value
Publication number	US-9519353-B2
Application number	US-41368309-A
Country	US
Kind code	B2
Filing date	Mar 30, 2009
Priority date	Mar 30, 2009
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

The invention relates to systems and or methodologies for enabling combined speech and touch inputs for observation symbol mappings. More particularly, the current innovation leverages the commonality of touch screen display text entry and speech recognition based text entry to increase the speed and accuracy of text entry via mobile devices. Touch screen devices often contain small and closely grouped keypads that can make it difficult for a user to press the intended character, by combining touch screen based text entry with speech recognition based text entry the aforementioned limitation can be overcome efficiently and conveniently.

First claim

What is claimed is:

1. A system for mobile device text entry, comprising:
memory storing computer readable instructions configured for executing a plurality of components; and
a processor, coupled to the memory and operable to execute the instructions, the plurality of components comprising:
a text recognition component that obtains a set of inputs from a text entry component, determines a set of digitized coordinates for the set of inputs, and produces a first set of observations based on a set of possible intended user inputs and the digitized coordinates, wherein the text entry component is a touch screen display;
a voice recognition component that obtains audio data from an audio capture component, and produces a second set of observations based on the audio data, wherein the audio capture component obtains the audio data concurrently as the text entry component obtains the set of inputs; and
a predictive component that combines the first set of observations and the second set of observations into a concatenated observation sequence, and generates a textual output based on the concatenated observation sequence, wherein the generated textual output includes a first predetermined character that is predicted by the predictive component based on (a) a substitute character received by the text entry component, the substitute character is different from the first predetermined character, and (b) pronunciation of the first predetermined character received by the audio capture component.

2. The system of claim 1 , wherein the touch screen display provides a virtual set of keys for a user to input a set of characters.

3. The system of claim 1 , wherein the user triggers an input via the text entry component that enables a secondary input mode, wherein the secondary input mode includes speech recognition.

4. The system of claim 1 wherein the predictive component generates the textual output based on the sets in the concatenated observation sequence by employing at least one of a Hidden Markov model and a Viterbi decoder that processes the concatenated observation sequence.

5. The system of claim 1 wherein the first predetermined character is a special character.

6. A method for mobile device text entry, comprising:
generating a first set of observations based on a set of possible intended user inputs and a determined set of digitized coordinates associated with the set of possible intended user inputs obtained via a text input device, wherein the text input device is a touch screen display;
generating a second set of observations based on user inputs obtained via an audio input device, wherein user inputs via the text input device and the audio input device are obtained concurrently;
combining the first set of observations and the second set of observations to produce a set of combined observations; and
producing a textual output by processing the set of combined observations, wherein the produced textual output includes a first predetermined character that is predicted based on (a) a substitute character received by the text input device, the substitute character is different from the first predetermined character, and (b) pronunciation of the first predetermined character received by the audio input device.

7. The method of claim 6 , wherein the touch screen display provides a graphical key wedge which enables users to select at least one character by touching the character on the display.

8. The method of claim 6 , further comprising triggering an input that enables an alternative input mode.

9. The method of claim 8 , wherein the alternative input mode includes speech recognition.

10. The method of claim 8 , further comprising enabling the user to access inputs via the alternative input mode that are not currently available.

11. The method of claim 6 wherein the textual output is produced by processing the set of combined observations employing at least one of a Hidden Markov model and a Viterbi decoder.

12. The method of claim 6 wherein the first predetermined character is a special character.

13. A system for mobile device text entry, comprising:
means for generating a first set of observations based on a set of possible intended user inputs and a determined set of digitized coordinates associated with the set of possible intended user inputs obtained via a touch screen display that provides a graphical key interface, wherein users select at least one unintended character by pressing at a location on the graphical key interface proximate to the location of the unintended character;
means for generating a second set of observations based on user inputs obtained via an audio capture device, wherein user inputs via the touch screen display and the audio capture device are obtained concurrently; and
means for combining the first set of observations and the second set of observations to produce a set of concatenated observations; and
means for producing a textual output based on the set of concatenated observations, wherein the produced textual output includes a first predetermined character that is predicted based on (a) a substitute character received by the graphical key interface, the substitute character is different from the first predetermined character, and (b) pronunciation of the first predetermined character received by the audio capture device.

14. The system of claim 13 , further comprising means for triggering an input that enables an alternative input mode, wherein the alternative input mode provides the user with input options that are not presently available.

15. The system of claim 13 wherein the textual output is produced by using at least one of a Viterbi decoder and a Hidden-Markov Model.

16. The system of claim 13 wherein the first predetermined character is a special character.

Assignees

• Willins Bruce
• Symbol Technologies Llc

Inventors

• Willins Bruce

Classifications

• G06F40/274

  Converting codes to words; Guess-ahead of partial word inputs · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• H04M1/271

  controlled by voice recognition · CPC title

• G10L15/20

  Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) · CPC title

• G06F3/0237 Primary

  using prediction or retrieval techniques · CPC title