System and method for initiating multi-modal speech recognition using a long-touch gesture

Patent metadata

Field	Value
Publication number	US-2016124706-A1
Application number	US-201414529766-A
Country	US
Kind code	A1
Filing date	Oct 31, 2014
Priority date	Oct 31, 2014
Publication date	May 5, 2016
Grant date	—

Abstract

A system, method and computer-readable storage devices are disclosed for multi-modal interactions with a system via a long-touch gesture on a touch-sensitive display. A system operating per this disclosure can receive a multi-modal input comprising speech and a touch on a display, wherein the speech comprises a pronoun. When the touch on the display has a duration longer than a threshold duration, the system can identify an object within a threshold distance of the touch, associate the object with the pronoun in the speech, to yield an association, and perform an action based on the speech and the association.

First claim

We claim:

1 . A method comprising:
receiving a multi-modal input comprising speech and a touch on a display; and when the touch on the display has a duration longer than a threshold duration:
identifying an object within a threshold distance of the touch;
associating the object with the pronoun in the speech, to yield an association; and
performing an action based on the speech and the association.

2 . The method of claim 1 , wherein the speech comprises a pronoun.

3 . The method of claim 1 , wherein a pronoun is implied in the speech.

4 . The method of claim 1 , wherein:
the display is presenting a computer-aided design program; and the action modifies a design within the computer-aided design program.

5 . The method of claim 4 , further comprising:
receiving a second touch on the display, wherein the action requires the second touch.

6 . The method of claim 1 , wherein the threshold duration is based on a context for the touch on the display.

7 . The method of claim 1 , wherein the threshold duration is based on a recognition certainty of a command recognized in the speech.

8 . The method of claim 2 , wherein the object is identified based, at least in part, on the pronoun.

9 . The method of claim 1 , wherein the speech of the multi-modal input is received simultaneously with initiation of the touch on the display.

10 . The method of claim 1 , wherein the speech of the multi-modal input is received after a duration of the touch on the display is determined to meet a long touch threshold.

11 . The method of claim 1 , wherein the speech of the multi-modal input is received after a duration of the touch on the display is determined to meet a press and hold threshold.

12 . A system comprising:
a processor; and a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising: receiving a multi-modal input comprising speech and a touch on a display, wherein the speech comprises a pronoun; and when the touch on the display has a duration longer than a threshold duration:
identifying an object within a threshold distance of the touch;
associating the object with the pronoun in the speech, to yield an association; and
performing an action based on the speech and the association.

13 . The system of claim 12 , wherein the threshold duration is based on a context for the touch on the display.

14 . The system of claim 12 , wherein the threshold duration is based on a recognition certainty of a command recognized in the speech.

15 . The system of claim 12 , wherein the object is identified based, at least in part, on the pronoun.

16 . The system of claim 12 , wherein the speech of the multi-modal input is received simultaneously with initiation of the touch on the display.

17 . The system of claim 12 , wherein the speech of the multi-modal input is received after a duration of the touch on the display is determined to meet a long touch threshold.

18 . The system of claim 12 , wherein the speech of the multi-modal input is received after a duration of the touch on the display is determined to meet a press and hold threshold.

19 . A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
receiving a multi-modal input comprising speech and a touch on a display, wherein the speech comprises a pronoun; and when the touch on the display has a duration longer than a threshold duration:
identifying an object within a threshold distance of the touch;
associating the object with the pronoun in the speech, to yield an association; and
performing an action based on the speech and the association.

20 . The computer-readable storage device of claim 19 , wherein:
the display is presenting a computer-aided design program; and the action modifies a design within the computer-aided design program.

Assignees

• At & T Ip I Lp

Inventors

• Vasilieff Brant J
• Ehlen Patrick
• Johnston Michael J

Classifications

• G10L2015/228

  of application context · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title