Contextual hotwords

Patent metadata

Field	Value
Publication number	US-10839803-B2
Application number	US-201916362831-A
Country	US
Kind code	B2
Filing date	Mar 25, 2019
Priority date	Dec 27, 2016
Publication date	Nov 17, 2020
Grant date	Nov 17, 2020

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for contextual hotwords are disclosed. In one aspect, a method, during a boot process of a computing device, includes the actions of determining, by a computing device, a context associated with the computing device. The actions further include, based on the context associated with the computing device, determining a hotword. The actions further include, after determining the hotword, receiving audio data that corresponds to an utterance. The actions further include determining that the audio data includes the hotword. The actions further include, in response to determining that the audio data includes the hotword, performing an operation associated with the hotword.

First claim

What is claimed is:

1. A computer-implemented method comprising:
determining, by a computing device, a current context associated with the computing device;
based on the current context associated with the computing device, determining, by the computing device and from among multiple different commands that the computing device is configured to execute, a command that is currently available for execution by a user to open an application that is not currently running on the computing device, wherein the multiple different commands other than the command are not currently available for execution by the user;
determining, by the computing device, a hotword that, when spoken by the user and detected by the computing device, instructs the computing device to execute the command, wherein each of the multiple different commands other than the command corresponds to a respective different hotword for triggering performance of a different respective operation other than opening the application that is not currently running on the computing device;
selecting, by the computing device and from among multiple hotword models, a hotword model that is configured to recognize audio of the hotword, wherein each of the multiple hotword models are configured to recognize audio of the respective hotword;
receiving, by the computing device, audio data of an utterance;
providing, by the computing device, the audio data of the utterance as an input to the hotword model without providing the audio data of the utterance as an input to the multiple hotword models other than the hotword model;
based on providing the audio data of the utterance as the input to the hotword model, determining, by the computing device, that the utterance includes the hotword; and
based on determining that the utterance includes the hotword, performing, by the computing device, the command by opening the application to commence running on the computing device.

2. The method of claim 1 , wherein determining that the utterance includes the hotword comprises determining that the utterance includes the hotword without performing speech recognition on the audio data.

3. The method of claim 1 , wherein the audio data only includes the hotword.

4. The method of claim 1 , further comprising:
receiving, by the computing device, additional audio data of an additional utterance that includes an additional hotword;
providing, by the computing device, the additional audio data of the additional utterance as an input to the hotword model without providing the additional audio data of the additional utterance as an input to the multiple hotword models other than the hotword model;
based on providing the additional audio data of the additional utterance as the input to the hotword model, determining, by the computing device, that the additional utterance does not include the hotword; and
bypassing, by the computing device, performing an additional command associated with the additional hotword.

5. The method of claim 1 , wherein each of the multiple hotword models are generated based on audio data of previous utterances that included a corresponding hotword.

6. The method of claim 1 , further comprising:
determining, by the computing device, that the command is no longer available for execution by the user;
determining, by the computing device and from among the multiple different commands that the computing device is configured to execute, that an additional command is available for execution by the user;
determining, by the computing device, an additional hotword that, when spoken by the user and detected by the computing device, instructs the computing device to execute the additional command;
selecting, by the computing device and from among the multiple hotword models, an additional hotword model that is configured to recognize audio of the additional hotword;
receiving, by the computing device, additional audio data of an additional utterance, wherein the additional utterance includes the hotword;
providing, by the computing device, the additional audio data of the additional utterance as an input to the additional hotword model without providing the audio data of the utterance as an input to the multiple hotword models other than the additional hotword model;
based on providing the additional audio data of the additional utterance as the input to the additional hotword model, determining, by the computing device, that the additional utterance does not include the additional hotword; and
based on determining that the additional utterance does not include the additional hotword, bypassing, by the computing device, performance of the additional command.

7. The method of claim 1 , wherein determining that the utterance includes the hotword comprises:
determining audio features of the audio data;
based on the audio features, determining a hotword confidence score that reflects a likelihood that the utterance includes the hotword; and
based on the hotword confidence score, determining that the utterance includes the hotword.

8. A system comprising:
one or more computers; and
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
determining, by a computing device, a current context associated with the computing device;
based on the current context associated with the computing device, determining, by the computing device and from among multiple different commands that the computing device is configured to execute, a command that is currently available for execution by a user to open an application that is not currently running on the computing device, wherein the multiple different commands other than the command are not currently available for execution by the user;
determining, by the computing device, a hotword that, when spoken by the user and detected by the computing device, instructs the computing device to execute the command, wherein each of the multiple different commands other than the command corresponds to a respective different hotword for triggering performance of a different respective operation other than opening the application that is not currently running on the computing device;
selecting, by the computing device and from among multiple hotword models, a hotword model that is configured to recognize audio of the hotword, wherein each of the multiple hotword models are configured to recognize audio of the respective hotword;
receiving, by the computing device, audio data of an utterance;
providing, by the computing device, the audio data of the utterance as an input to the hotword model without providing the audio data of the utterance as an input to the multiple hotword models other than the hotword model;
based on providing the audio data of the utterance as the input to the hotword model, determining, by the computing device, that the utterance includes the hotword; and
based on determining that the utterance includes the hotword, performing, by the computing device, the command by opening the application to commence running on the computing device.

9. The system of claim 8 , wherein determining that the utterance includes the hotword comprises determining that the utterance includes the hotword without performing speech recognition on the audio data.

10. The system of claim 8 , wherein the audio data only includes the hotword.

11. The system of claim 8 , wherein the operations further comprise:
receiving, by the computing device, additional audio data of an additional utterance that includes an additional hotword;
providing, by the computing devic

Assignees

• Google Llc

Inventors

• Hughes Christopher Thaddeus
• Lopez Moreno Ignacio
• Kracun Aleksandar

Classifications

• G10L2015/228

  of application context · CPC title

• G10L2015/226

  using non-speech characteristics · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L2015/088

  Word spotting · CPC title

• G10L15/20

  Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) · CPC title