System and method for detecting errors in interactions with a voice-based digital assistant

Patent metadata

Field	Value
Publication number	US-9633674-B2
Application number	US-201414297448-A
Country	US
Kind code	B2
Filing date	Jun 5, 2014
Priority date	Jun 7, 2013
Publication date	Apr 25, 2017
Grant date	Apr 25, 2017

Abstract

The method is performed at an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors. A speech input containing a request is received from a user. At least one action in furtherance of satisfying the request is performed. A user interaction is detected, such as a speech input to a digital assistant or a physical interaction with a device. It is determined whether the user interaction is indicative of a problem in the performing of the at least one action. Upon determining that the user interaction is indicative of a problem, information relating to the request is stored in a repository for error analysis.

First claim

What is claimed is:

1. A method for operating an intelligent automated assistant, comprising:
at an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors:
receiving, from a user, a speech input containing a request;
performing at least one action in furtherance of satisfying the request;
detecting a user interaction;
determining whether the user interaction is indicative of a problem in the performing of the at least one action, wherein the user interaction includes a shaking of the electronic device;
upon determining that the user interaction is indicative of a problem, storing information relating to the request in a repository for error analysis.

2. The method of claim 1 , wherein detecting the user interaction comprises detecting an additional speech input, and determining whether the user interaction is indicative of a problem comprises determining that the additional speech input indicates dissatisfaction with the at least one action.

3. The method of claim 2 , wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes at least one predefined word.

4. The method of claim 2 , wherein determining whether the additional speech input indicates dissatisfaction includes determining a volume of the additional speech input.

5. The method of claim 4 , wherein the additional speech input indicates dissatisfaction when the volume of the additional speech input is above a predefined threshold.

6. The method of claim 2 , wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input contains prosodic indications of frustration.

7. The method of claim 2 , wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes substantially the same words as the first speech input.

8. The method of claim 1 , wherein the user interaction comprises one or more of:
a predefined motion of the device;
a selection of an affordance;
a termination of a dialog session with the intelligent automated assistant; or
a rejection of a proposed task.

9. The method of claim 1 , further comprising, upon determining that the user interaction is indicative of a problem:
providing a prompt requesting the user to confirm whether there was a problem in the performing of the at least one action; and
receiving, from the user, a confirmation or a disconfirmation of whether there was a problem in the performing of the at least one action.

10. The method of claim 1 , wherein the repository includes a plurality of entries from a plurality of users, the method further comprising:
analyzing the repository to identify a set of entries, each entry of the set of entries having one or more similar characteristics indicative of an error; and
adjusting one or more of a speech-to-text module and a natural language processing module based on the set of entries so as to reduce reproduction of the error.

11. The method of claim 1 , further comprising:
upon determining that the user interaction indicates an absence of a problem, performing at least one of:
avoiding to store the information relating to the request in the repository, and
removing the information relating to the request from the repository.

12. The method of claim 11 , wherein performing at least one of: avoiding to store the information relating to the request in the repository, and removing the information relating to the request from the repository comprises:
if the information relating to the request is absent in the repository, avoiding to store information relating to the request in the repository; or
if the information relating to the request is stored in the repository, removing the information relating to the request from the repository.

13. An electronic device, comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving, from a user, a speech input containing a request;
performing at least one action in furtherance of satisfying the request;
detecting a user interaction;

determining whether the user interaction is indicative of a problem in the performing of the at least one action, wherein the user interaction includes a shaking of the electronic device;
upon determining that the user interaction is indicative of a problem, storing information relating to the request in a repository for error analysis.

14. The electronic device of claim 13 , wherein detecting the user interaction comprises detecting an additional speech input, and determining whether the user interaction is indicative of a problem comprises determining that the additional speech input indicates dissatisfaction with the at least one action.

15. The electronic device of claim 14 , wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes at least one predefined word.

16. The electronic device of claim 14 , wherein determining whether the additional speech input indicates dissatisfaction includes determining a volume of the additional speech input.

17. The electronic device of claim 16 , wherein the additional speech input indicates dissatisfaction when the volume of the additional speech input is above a predefined threshold.

18. The electronic device of claim 14 , wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input contains prosodic indications of frustration.

19. The electronic device of claim 14 , wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes substantially the same words as the first speech input.

20. The electronic device of claim 13 , wherein the user interaction comprises one or more of:
a predefined motion of the device;
a selection of an affordance;
a termination of a dialog session with the intelligent automated assistant; or
a rejection of a proposed task.

21. The electronic device of claim 13 , further comprising:
upon determining that the user interaction indicates an absence of a problem, performing at least one of:
avoiding to store the information relating to the request in the repository, and
removing the information relating to the request from the repository.

22. The electronic device of claim 21 , wherein performing at least one of: avoiding to store the information relating to the request in the repository, and removing the information relating to the request from the repository comprises:
if the information relating to the request is absent in the repository, avoiding to store information relating to the request in the repository; or
if the information relating to the request is stored in the repository, removing the information relating to the request from the repository.

23. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors and memory, cause the device to:
receiv

Assignees

• Apple Inc

Inventors

• Sinha Anoop K

Classifications

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• H04M1/72519

  Electricity · mapped topic

• G10L25/00 Primary

  Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 (muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34) · CPC title

• H04M2250/74

  with voice recognition means · CPC title

• G10L2015/227

  of the speaker; Human-factor methodology · CPC title