Automatic speech recognition based on user feedback

Patent metadata

Field	Value
Publication number	US-10446141-B2
Application number	US-201514591754-A
Country	US
Kind code	B2
Filing date	Jan 7, 2015
Priority date	Aug 28, 2014
Publication date	Oct 15, 2019
Grant date	Oct 15, 2019

Abstract

Systems and processes for processing speech in a digital assistant are provided. In one example process, a first speech input can be received from a user. The first speech input can be processed using a first automatic speech recognition system to produce a first recognition result. An input indicative of a potential error in the first recognition result can be received. The input can be used to improve the first recognition result. For example, the input can include a second speech input that is a repetition of the first speech input. The second speech input can be processed using a second automatic speech recognition system to produce a second recognition result.

First claim

What is claimed is:

1. A method for processing speech in a digital assistant, the method comprising:
at an electronic device with a processor and memory storing one or more programs for execution by the processor:
receiving, from a network interface, a first speech input;
processing the first speech input using a first automatic speech recognition system to produce a first speech recognition result;
performing a first task corresponding to a first user intent determined from the first speech recognition result;
upon performing the first task, receiving, from the network interface, an input representing a rejection of the first task;
in response to receiving the input, providing a prompt seeking a repetition of at least a portion of the first speech input;
receiving, from the network interface, a second speech input;
in accordance with the received input representing a rejection of the first task, processing the second speech input using a second automatic speech recognition system to produce a second speech recognition result, wherein the first automatic speech recognition system includes one or more speech recognition models, and the second automatic speech recognition system includes one or more speech recognition models that are different from the one or more speech recognition models of the first automatic speech recognition system;
determining a combined speech recognition result based on the first speech recognition result and the second speech recognition result; and
performing a second task corresponding to a second user intent determined from the combined speech recognition result.

2. The method of claim 1 , wherein the input is a speech input that includes a predetermined utterance.

3. The method of claim 1 , wherein the input comprises a selection of an affordance.

4. The method of claim 1 , wherein at least a portion of text of the first speech recognition result is displayed on the electronic device, and wherein the input comprises a selection of at least a portion of the displayed text.

5. The method of claim 1 , further comprising:
in accordance with receiving the input, identifying a portion of the first speech input corresponding to a potential error in the first speech recognition result.

6. The method of claim 5 , wherein processing the first speech input using the first automatic speech recognition system includes determining a confidence measure of each word in a text of the first speech recognition result, and wherein the portion of the first speech input associated with the potential error is identified based on the confidence measure of each word in the text.

7. The method of claim 5 , wherein the prompt includes a request to repeat the identified portion of the first speech input corresponding to the potential error.

8. The method of claim 1 , wherein the combined result is determined by performing automatic speech recognition system combination using the first speech recognition result and the second speech recognition result.

9. The method of claim 1 , wherein the second automatic speech recognition system is associated with a greater computation cost than the first automatic speech recognition system in order to achieve greater accuracy.

10. A method for processing speech in a digital assistant, the method comprising:
at an electronic device with a processor and memory storing one or more programs for execution by the processor:
receiving an input containing user speech;
processing the input using a first automatic speech recognition system to produce a first speech recognition result;
performing a first task corresponding to a first user intent determined from the first speech recognition result;
upon performing the first task, receiving a second input representing a rejection of the first task;
in response to receiving the second input, processing at least a portion of the audio signal using a second automatic speech recognition system to produce a second speech recognition result, wherein the first automatic speech recognition system includes one or more speech recognition models, and the second automatic speech recognition system includes one or more speech recognition models that are different from the one or more speech recognition models of the first automatic speech recognition system;
determining a combined speech recognition result based on the first speech recognition result and the second speech recognition result; and
performing a second task corresponding to a second user intent determined from the combined speech recognition result.

11. The method of claim 10 , wherein an error rate of the second automatic speech recognition system is lower than an error rate of the first automatic speech recognition system.

12. The method of claim 10 , wherein a latency of the second automatic speech recognition system is greater than a latency of the first automatic speech recognition system.

13. The method of claim 10 , wherein the combined result is determined by performing automatic speech recognition system combination using the first speech recognition result and the second speech recognition result.

14. The method of claim 13 , wherein performing automatic speech recognition system combination comprises implementing at least one of recognition output voting error reduction, cross-adaptation, confusion network combination, and lattice combination.

15. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs comprising instructions for:
receiving, from a network interface, a first speech input;
processing the first speech input using a first automatic speech recognition system to produce a first speech recognition result;
performing a first task corresponding to a first user intent determined from the first speech recognition result;
receiving, from the network interface, a second speech input;
determining whether a phonemic transcription of the second speech input has an error rate that is less than a predetermined value when compared against a phonemic transcription of a corresponding portion of the first speech input;
in response to determining that the phonemic transcription of the second speech input has an error rate that is less than the predetermined value when compared against the phonemic transcription of a corresponding portion of the first speech input, processing the second speech input using a second automatic speech recognition system to produce a second speech recognition result; and
performing a second task corresponding to a second user intent determined based on the second speech recognition result.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the first automatic speech recognition system includes one or more speech recognition models, and the second automatic speech recognition system includes one or more speech recognition models that are different from the one or more speech recognition models of the first automatic speech recognition system.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the one or more programs further including instructions for:
determining a combined speech recognition result based on the first speech recognition result and the second speech recognition result, wherein the second user intent is determined further based on the combined speech recognition result.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the combined speech recognition

Assignees

• Apple Inc

Inventors

• Krishnamoorthy Mahesh
• Paulik Matthias

Classifications

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/01

  Assessment or evaluation of speech recognition systems · CPC title

• G10L15/32

  Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems · CPC title

• G10L2015/025

  Phonemes, fenemes or fenones being the recognition units · CPC title

• G10L15/02

  Feature extraction for speech recognition; Selection of recognition unit · CPC title