Reducing digital assistant latency when a language is incorrectly determined

Patent metadata

Field	Value
Publication number	US-11475884-B2
Application number	US-201916548567-A
Country	US
Kind code	B2
Filing date	Aug 22, 2019
Priority date	May 6, 2019
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

Systems and processes for operating an intelligent automated assistant are provided. An example process includes causing a first recognition result for a received natural language speech input to be displayed, where the first recognition result is in a first language and a second recognition result for the received natural language speech input is available for display responsive to receiving input indicative of user selection of the first recognition result, the second recognition result being in a second language. The example process further includes receiving the input indicative of user selection of the first recognition result and in response to receiving the input indicative of user selection of the first recognition result, causing the second recognition result to be displayed.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:
determine a first likelihood that a received natural language speech input is in a first language;
in accordance with a determination that the first likelihood is less than a threshold, cause a first recognition result for the received natural language speech input to be displayed concurrently with a visual indication of another available recognition result, wherein:
the first recognition result is in the first language; and
a second recognition result for the received natural language speech input is available for display responsive to receiving input indicative of user selection of the first recognition result, the second recognition result being in a second language;

while displaying the first recognition result concurrently with the visual indication, receive the input indicative of user selection of the first recognition result; and
in response to receiving the input indicative of user selection of the first recognition result, cause the second recognition result to be displayed.

2. The non-transitory computer-readable storage medium of claim 1 , wherein:
the electronic device includes a display;
causing the first recognition result to be displayed includes displaying the first recognition result at a first location on the display; and
the input indicative of user selection of the first recognition result includes a user gesture at the first location.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the second recognition result is available for display without receiving natural language speech input other than the received natural language speech input.

4. The non-transitory computer-readable storage medium of claim 1 , wherein causing the second recognition result to be displayed includes causing the second recognition result to be displayed adjacent to the displayed first recognition result.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:
in accordance with causing the second recognition result to be displayed:
receive an input indicative of user selection of the displayed second recognition result;
in response to receiving the input indicative of user selection of the displayed second recognition result, initiate a task based on the second recognition result; and
provide a result based on the task.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:
receive the natural language speech input;
process the natural language speech input using a language recognizer for the first language, including determining, based on the natural language speech input, the first recognition result; and
while processing the natural language speech input using the language recognizer for the first language:
process the natural language speech input using a language recognizer for the second language, including determining, based on the natural language speech input, the second recognition result.

7. The non-transitory computer-readable storage medium of claim 6 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:
determine a second likelihood that the natural language speech input is the second language; and
determine whether the first likelihood or the second likelihood exceeds a second threshold.

8. The non-transitory computer-readable storage medium of claim 7 , wherein:
determining the first likelihood includes determining the first likelihood while processing the natural language speech input using the language recognizer for the first language; and
determining the second likelihood includes determining the second likelihood while processing the natural language speech input using the language recognizer for the first language.

9. The non-transitory computer-readable storage medium of claim 7 , wherein causing the first recognition result to be displayed is performed in accordance with a determination that the first likelihood exceeds the second threshold and in accordance with a determination that the second likelihood does not exceed the second threshold.

10. The non-transitory computer-readable storage medium of claim 7 , wherein the visual indication includes an underlining of the first recognition result.

11. The non-transitory computer-readable storage medium of claim 10 , wherein causing the result visual indication of another available recognition result to be displayed is performed in accordance with determining that the first likelihood is greater than the second likelihood.

12. The non-transitory computer-readable storage medium of claim 7 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:
select, from a plurality of language classifiers, a language classifier for the first language and a language classifier for the second language based on first context information; and
wherein:
determining the first likelihood includes determining the first likelihood using the language classifier for the first language; and
determining the second likelihood includes determining the second likelihood using the language classifier for the second language.

13. The non-transitory computer-readable storage medium of claim 6 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:
select, from a plurality of language recognizers, the language recognizer for the first language and the language recognizer for the second language based on second context information associated with a user.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the second context information includes one or more respective languages of one or more keyboards of the user.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the second context information includes one or more respective languages of one or more communications associated with the user.

16. The non-transitory computer-readable storage medium of claim 13 , wherein the second context information includes a location of the user.

17. The non-transitory computer-readable storage medium of claim 1 , wherein causing the first recognition result to be displayed is based on a user setting.

18. An electronic device, comprising:
one or more processors;
memory; and
one or more programs stored in the memory, the one or more programs including instructions for:
determining a first likelihood that a received natural language speech input is in a first language;
in accordance with a determination that the first likelihood is less than a threshold, causing a first recognition result for the received natural language speech input to be displayed concurrently with a visual indication of another available recognition result, wherein:
the first recognition result is in the first language; and
a second recognition result for the received natural language speech input is available f

Assignees

• Apple Inc

Inventors

• Ghoshal Arnab
• Hsiao Roger
• Amand Gorm
• Coffman Patrick L
• Young Mary

Classifications

• G10L15/005 Primary

  Language recognition · CPC title

• G10L2015/221

  Announcement of recognition results · CPC title

• G10L15/32

  Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems · CPC title

• G10L15/1815 Primary

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title