Multi-level speech recognition

Patent metadata

Field	Value
Publication number	US-9305554-B2
Application number	US-201414333463-A
Country	US
Kind code	B2
Filing date	Jul 16, 2014
Priority date	Jul 17, 2013
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

A method and device for recognizing an utterance. The method includes transmitting context data associated with a first device to a second device. A first speech recognition model is received from the second device. The first speech recognition model is a subset of a second speech recognition model present at the second device. The first speech recognition model is based on the context data. It is determined whether the utterance can be recognized at the first device based on the first speech recognition model. If the utterance cannot be recognized at the first device, then at least a portion of the utterance is sent to the second device. If the utterance can be recognized at the first device, then an action associated with the recognized utterance is performed.

First claim

What is claimed is:

1. A method comprising:
obtaining, by a first electronic device, context data comprising at least one of location, time and activity, wherein the context data is associated with the first electronic device;
transmitting, by the first electronic device, the context data to a second electronic device;
receiving, by the first electronic device, a first speech recognition model, the first speech recognition model being a subset of a second speech recognition model present at the second electronic device, wherein the first speech recognition model is selected based on the context data;
determining, by the first electronic device, whether an utterance can be recognized by a speech recognition process, wherein the speech recognition process is performed by the first electronic device and uses the first speech recognition model;
in response to determining that the utterance cannot be recognized by the speech recognition process, sending, by the first electronic device, at least a portion of the utterance to the second electronic device; and
in response to determining that the utterance can be recognized by the speech recognition process, causing, by the first electronic device, performance of an action associated with the utterance.

2. The method of claim 1 , wherein the second electronic device comprises a higher level of processing capability than the first electronic device.

3. The method of claim 1 , wherein:
the first speech recognition model includes at least one of: a first language model and a first acoustic model; and
the second speech recognition model includes at least one of: a second language model and a second acoustic model.

4. The method of claim 1 wherein the action associated with the utterance is performed locally at the first electronic device.

5. The method of claim 1 further comprising:
upon a determination that the first electronic device is near a third electronic device, receiving, by the first electronic device, a third speech recognition model from the third electronic device; and
transmitting, by the first electronic device, speech recognition information to the third electronic device, wherein the speech recognition information is generated by recognizing an utterance that is received by the third electronic device.

6. The method of claim 1 , wherein:
the determining whether the utterance can be recognized by the speech recognition process comprises determining whether a threshold value is exceeded; and
the threshold value is based on at least one of: an estimated word error rate, a length of an utterance, presence of keywords in an utterance, availability of a network connection, prior history of processing an utterance, and a processing capability of the first electronic device.

7. The method of claim 1 , further comprising:
receiving, by the first electronic device, speech recognition information that is based on selective pre-processing of speech recognition, wherein the selective pre-processing is based on the context data.

8. The method of claim 4 , further comprising determining, by the first electronic device, a particular electronic device that is in a vicinity of the first electronic device and that is capable of executing the action.

9. The method of claim 1 , wherein:
the speech recognition process is configured to occur within a hierarchy of electronic devices and includes a guaranteed order of execution of actions; and
the guaranteed order of execution of actions is based on using a gateway configured to determine order of action execution.

10. The method of claim 1 , wherein:
the speech recognition process includes speech decoding;
the speech decoding is selectively passed between the first electronic device and one or more electronic devices that have a higher processing level than the first electronic device; and
the speech decoding is processed sequentially or hierarchically.

11. The method of claim 10 , wherein the speech decoding continues until a particular processing level is matched.

12. The method of claim 1 , further comprising:
determining, using access patterns, at least one automatic speech recognition (ASR) implementation, wherein the action is executed based on multiple language models and multiple acoustic models, and wherein multiple actionable commands are published by and are subscribed to by one or more networked devices.

13. The method of claim 1 , wherein the first electronic device is a mobile electronic device, a smart appliance device, smart television device, or a smart home system.

14. A first electronic device comprising:
a processor device configured to obtain context data comprising at least one of location, time and activity, wherein the context data is associated with the first electronic device;
a transmitter configured to transmit the context data to a second electronic device;
a microphone configured to capture an utterance; and
a speech processor configured to: receive a first speech recognition model, wherein the first speech recognition model is a subset of a second speech recognition model present at the second electronic device, and the first speech recognition model is selected based on the context data, determine whether the utterance can be recognized by a speech recognition process, wherein the speech recognition process is performed by the speech processor and uses the first speech recognition model, send at least a portion of the utterance to the second electronic device in response to a determination that the utterance cannot be recognized by the speech recognition process, and upon a determination that the utterance can be recognized by the speech recognition process, causing by the first electronic device, performance of an action associated with the utterance.

15. The first electronic device of claim 14 , wherein the second electronic device comprises a higher level of processing capability than the first electronic device.

16. The first electronic device of claim 14 , wherein:
the first speech recognition model includes at least one of: a first language model and a first acoustic model; and
the second speech recognition model includes at least one of: a second language model and a second acoustic model.

17. The first electronic device of claim 14 , wherein:
the speech processor is configured to:
receive a third speech recognition model from a third electronic device; and when the first electronic device is near the third electronic device, process, using the third speech recognition model, an utterance received by the third electronic device;

the transmitter is configured to transmit speech recognition information to the third electronic device; and
the speech recognition information is generated by recognizing the utterance received by the third electronic device.

18. The first electronic device of claim 14 , wherein:
the speech processor is configured to determine whether a threshold value is exceeded; and
the threshold value is based on at least one of: an estimated word error rate, a length of an utterance, presence of keywords in an utterance, availability of a network connection, prior history of processing an utterance, and a processing capability of the first electronic device.

19. The first electronic device of claim 16 , wherein:
the speech processor is configured to receive speech recognition information that is based on selective pre-processing speech recognition; and
the selective pre-processing is based on the context data.

20. The first electro

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jagatheesan Arun
• Lee Juhan
• Ahnn Jong Hoon

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G10L15/183 Primary

  using context dependencies, e.g. language models · CPC title

• G10L2015/228

  of application context · CPC title

• G10L15/32 Primary

  Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems · CPC title