Speech recognition candidate selection based on non-acoustic input

Patent metadata

Field	Value
Publication number	US-2016140955-A1
Application number	US-201414540527-A
Country	US
Kind code	A1
Filing date	Nov 13, 2014
Priority date	Nov 13, 2014
Publication date	May 19, 2016
Grant date	—

Abstract

A method includes the following steps. A speech input is received. At least two speech recognition candidates are generated from the speech input. A scene related to the speech input is observed using one or more non-acoustic sensors. The observed scene is segmented into one or more regions. One or more properties for the one or more regions are computed. One of the speech recognition candidates is selected based on the one or more computed properties of the one or more regions.

First claim

1 .- 12 . (canceled)

13 . An apparatus, comprising:
a memory; and a processor operatively coupled to the memory and configured to:
receive a speech input;
generate at least two speech recognition candidates from the speech input;
observe a scene related to the speech input using one or more non-acoustic sensors;
segment the observed scene into one or more regions;
compute one or more properties for the one or more regions; and
select one of the speech recognition candidates based on the one or more computed properties of the one or more regions.

14 . The apparatus of claim 13 , wherein each speech recognition candidate comprises one of a word and a phrase.

15 . The apparatus of claim 13 , wherein the speech recognition candidate is selected based on a comparison of the one or more computed properties of the one or more regions with the two or more speech recognition candidates generated based on the speech input.

16 . The apparatus of claim 15 , further comprising interpreting the selected speech recognition candidate as a portion of an action command.

17 . The apparatus of claim 13 , wherein the one or more properties comprises a color, a shape and a textual label.

18 . The apparatus of claim 13 , wherein computing one or more properties of the one or more regions comprises determining one or more spatial relations between the one or more regions.

19 . The apparatus of claim 13 , wherein computing one or more properties of the one or more regions comprises detecting one or more gestures and associating the one or more detected gestures with at least a portion of the one or more regions.

20 . An article of manufacture comprising a computer readable storage medium for storing computer readable program code which, when executed, causes a computer to:
receive a speech input; generate at least two speech recognition candidates from the speech input; observe a scene related to the speech input using one or more non-acoustic sensors; segment the observed scene into one or more regions; compute one or more properties for the one or more regions; and select one of the speech recognition candidates based on the one or more computed properties of the one or more regions.

21 . The article of claim 20 , wherein each speech recognition candidate comprises one of a word and a phrase.

22 . The article of claim 20 , wherein the speech recognition candidate is selected based on a comparison of the one or more computed properties of the one or more regions with the two or more speech recognition candidates generated based on the speech input.

23 . The article of claim 22 , further comprising interpreting the selected speech recognition candidate as a portion of an action command.

24 . The article of claim 20 , wherein the one or more properties comprises a color, a shape and a textual label.

25 . The article of claim 20 , wherein computing one or more properties of the one or more regions comprises determining one or more spatial relations between the one or more regions.

26 . The article of claim 20 , wherein computing one or more properties of the one or more regions comprises detecting one or more gestures and associating the one or more detected gestures with at least a portion of the one or more regions.

Assignees

• IBM

Inventors

• Connell Ii Jonathan H
• Marcheret Etienne

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L15/08 Primary

  Speech classification or search · CPC title

• G10L17/22

  Interactive procedures; Man-machine interfaces · CPC title

• G10L15/25

  using position of the lips, movement of the lips or face analysis · CPC title