Voice command processing for locked devices

Patent metadata

Field	Value
Publication number	US-11862174-B2
Application number	US-202117209596-A
Country	US
Kind code	B2
Filing date	Mar 23, 2021
Priority date	May 29, 2018
Publication date	Jan 2, 2024
Grant date	Jan 2, 2024

Abstract

Techniques for processing voice commands from a locked device are described. A voice command received by a locked device is stored, a prompt requesting that the device be unlocked is generated, and the voice command is processed automatically after the device is unlocked. Thus, the system processes the voice command without the user repeating the voice command. In addition, the system may process certain voice commands even when the device is locked. For example, a whitelist filter compares an intent associated with the voice command to whitelisted intents from a whitelist database before the intent is dispatched to a speechlet, and intents included in the whitelist database are processed normally. Thus, the system performs certain voice commands while the device is locked, while other voice commands may be automatically processed after the device is unlocked without the user repeating the voice command.

First claim

What is claimed is:

1. A computer-implemented method comprising:
capturing, by at least one microphone of a first device in a locked state, audio representing an utterance;
sending, from the first device to a remote system while the first device is in the locked state, audio data representing the utterance;
determining, by the remote system, that the first device is in the locked state;
performing, by the remote system, speech processing on the audio data to determine speech processing results data, wherein the speech processing includes natural language understanding (NLU) processing and the speech processing results data includes first NLU results data, wherein the first NLU results data includes a plurality of potential intents associated with a respective confidence score;
determining that a first potential intent of the plurality of potential intents corresponds to an action that is not permitted to be performed while the first device is in the locked state, the first potential intent being associated with a first confidence score representing a highest confidence among the plurality of potential intents;
determining that a second potential intent of the plurality of potential intents corresponds to an action that is permitted to be performed while the first device is in the locked state, the second potential intent being associated with a second confidence score representing a lower confidence than the first confidence score;
sending, from the remote system to a speechlet component, data indicating the second potential intent;
receiving, by the remote system from the speechlet component, response data;
and
sending, from the remote system to the first device, data corresponding to execution of the action,
wherein the data corresponding to execution of the action is based, at least in part, on the response data.

2. The computer-implemented method of claim 1 , wherein the data corresponding to execution of the action includes a command for the first device to perform the action.

3. The computer-implemented method of claim 1 , further comprising:
sending, from the first device to the remote system, state data indicating the first device is in the locked state.

4. The computer-implemented method of claim 3 , further comprising:
sending, from the remote system to the speechlet component, the state data.

5. The computer-implemented method of claim 1 , wherein determining that the second potential intent of the plurality of intents corresponds to an action that is permitted to be performed while the first device is in the locked state is based at least in part on user profile data.

6. The computer-implemented method of claim 1 , further comprising:
determining, by the first device while in the locked state, that the audio comprises a wakeword.

7. The computer implemented method of claim 1 , further comprising:
determining, by the remote system, whether the speech processing results data refers to the speechlet component,
wherein determining that the second potential intent of the plurality of intents corresponds to an action that is permitted to be performed while the first device is in the locked state is based on determining that the second potential intent refers to the speechlet component.

8. A system comprising:
at least one processor; and
memory comprising instructions that, when executed by the at least one processor, cause the system to:
capture, by at least one microphone of a first device in a locked state, audio representing an utterance;
send, from the first device to a remote system while the first device is in the locked state, audio data representing the utterance;
determine, by the remote system, that the first device is in the locked state;
perform, by the remote system, speech processing on the audio data to determine speech processing results data, wherein the speech processing includes natural language understanding (NLU) processing and the speech processing results data includes first NLU results data, wherein the first NLU results data includes a plurality of potential intents associated with a respective confidence score;
determine that a first potential intent of the plurality of potential intents corresponds to an action that is not permitted to be performed while the first device is in the locked state, the first potential intent being associated with a first confidence score representing a highest confidence among the plurality of potential intents;
determine that a second potential intent of the plurality of potential intents corresponds to an action that is permitted to be performed while the first device is in the locked state, the second potential intent being associated with a second confidence score representing a lower confidence than the first confidence score;

send, from the remote system to a speechlet component, data, indicating the second potential intent;
receive, by the remote system from the speechlet component, response data;
and
send, from the remote system to the first device, data corresponding to execution of the action,
wherein the data corresponding to execution of the action is based, at least in part, on the response data.

9. The system of claim 8 , wherein the data corresponding to execution of the action includes a command for the first device to perform the action.

10. The system of claim 8 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
send, from the first device to the remote system, state data indicating the first device is in the locked state.

11. The system of claim 8 , wherein determination, that the second potential intent of the plurality of intents corresponds to an action that is permitted to be performed while the first device is in the locked state is based at least in part on user profile data.

12. The system of claim 8 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine, by the first device while in the locked state, that the audio comprises a wakeword.

13. A computer-implemented method comprising:
capturing, by at least one microphone of a first device in a locked state, audio representing an utterance;
sending, from the first device to a remote system while the first device is in the locked state, audio data representing the utterance;
determining, by the remote system, that the first device is in the locked state;
performing, by the remote system, speech processing on the audio data to determine speech processing results data, wherein the speech processing includes natural language understanding (NLU) processing and the speech processing results data includes first NLU results data, wherein the first NLU results data includes a plurality of potential intents associated with a respective confidence score;
determining that a first potential intent of the plurality of potential intents corresponds to an action that is not permitted to be performed while the first device is in the locked state, the first potential intent being associated with a first confidence score representing a highest confidence among the plurality of potential intents;
determining that a second potential intent of the plurality of potential intents corresponds to an action that is permitted to be performed while the first device is in the locked state, the second potential intent being associated with a second confidence score representing a lower confidence than the first confidence score;
sending, from the remote system to a speechlet component, data indicating the second potential intent;
rece

Assignees

• Amazon Tech Inc

Inventors

• Wang Haitang
• Vachhani Ankur Narendra Bhai

Classifications

• G10L17/00 Primary

  Speaker identification or verification techniques · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F21/31 Primary

  User authentication · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L17/22

  Interactive procedures; Man-machine interfaces · CPC title