Reducing the need for manual start/end-pointing and trigger phrases

Patent metadata

Field	Value
Publication number	US-9715875-B2
Application number	US-201414502737-A
Country	US
Kind code	B2
Filing date	Sep 30, 2014
Priority date	May 30, 2014
Publication date	Jul 25, 2017
Grant date	Jul 25, 2017

Abstract

Systems and processes for selectively processing and responding to a spoken user input are provided. In one example, audio input containing a spoken user input can be received at a user device. The spoken user input can be identified from the audio input by identifying start and end-points of the spoken user input. It can be determined whether or not the spoken user input was intended for a virtual assistant based on contextual information. The determination can be made using a rule-based system or a probabilistic system. If it is determined that the spoken user input was intended for the virtual assistant, the spoken user input can be processed and an appropriate response can be generated. If it is instead determined that the spoken user input was not intended for the virtual assistant, the spoken user input can be ignored and/or no response can be generated.

First claim

What is claimed is:

1. A method for operating a virtual assistant on an electronic device, the method comprising:
receiving, at the electronic device, an audio input;
monitoring the audio input to identify a first spoken user input;
identifying the first spoken user input in the audio input;
determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input, wherein the contextual information comprises a determined distance between a user and the electronic device when the first spoken user input was received, the determined distance being based on the first spoken user input, wherein determining whether to respond to the first spoken user input comprises
calculating a likelihood score that the virtual assistant should respond to the first spoken user input based on the contextual information associated with the first spoken user input,
comparing the likelihood score to a threshold value,
decreasing the likelihood score in response to the distance being greater than a threshold distance, and
increasing the likelihood score in response to the distance being less than the threshold distance;

in response to a determination to respond to the first spoken user input:
generating a response to the first spoken user input; and
monitoring the audio input to identify a second spoken user input; and

in response to a determination not to respond to the first spoken user input, monitoring the audio input to identify the second spoken user input without generating the response to the first spoken user input.

2. The method of claim 1 , wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input excludes identifying one or more predetermined words at the start of the first spoken user input.

3. The method of claim 1 , wherein generating the response to the first spoken user input comprises one or more of:
performing speech-to-text conversion on the first spoken user input;
determining a user intent sed on the first spoken user input;
determining a task to be performed based on the first spoken user input;
determining a parameter for the task to be performed based on the first spoken user input;
performing the task to be performed;
displaying a text response to the first spoken user input; and
outputting an audio response to the first spoken user input.

4. The method of claim 1 , wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input comprises:
evaluating one or more conditional rules that depend on the contextual information associated with the first spoken user input.

5. The method of claim 1 , further comprising determining the distance between the user and the electronic device.

6. The method of claim 5 , wherein determining the distance between the user and the electronic device is based on distance data provided by a proximity sensor of the electronic device.

7. A non-transitory computer-readable storage medium comprising instructions for:
receiving an audio input;
monitoring the audio input to identify a first spoken user input;
identifying the first spoken user input in the audio input;
determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input, wherein the contextual information comprises a determined distance between a user and the electronic device when the first spoken user input was received, the determined distance being based on the first spoken user input, wherein determining whether to respond to the first spoken user input comprises
calculating a likelihood score that the virtual assistant should respond to the first spoken user input based on the contextual information associated with the first spoken user input,
comparing the likelihood score to a threshold value,
decreasing the likelihood score in response to the distance being greater than a threshold distance, and
increasing the likelihood score in response to the distance being less than the threshold distance;

responsive to a determination to respond to the first spoken user input:
generating a response to the first spoken user input; and
monitoring the audio input to identify a second spoken user input; and

responsive to a determination not to respond to the first spoken user input, monitoring the audio input to identify the second spoken user input without generating the response to the first spoken user input.

8. A system comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving an audio input;
monitoring the audio input to identify a first spoken user input;
identifying the first spoken user input in the audio input;
determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input, wherein the contextual information comprises a determined distance between a user and the electronic device when the first spoken user input was received, the determined distance being based on the first spoken user input, wherein determining whether to respond to the first spoken user input comprises
calculating a likelihood score that the virtual assistant should respond to the first spoken user input based on the contextual information associated with the first spoken user input,
comparing the likelihood score to a threshold value,
decreasing the likelihood score in response to the distance being greater than a threshold distance, and
increasing the likelihood score in response to the distance being less than the threshold distance;

responsive to a determination to respond to the first spoken user input:
generating a response to the first spoken user input; and
monitoring the audio input to identify a second spoken user input; and

responsive to a determination not to respond to the first spoken user input, monitoring the audio input to identify the second spoken user input without generating the response to the first spoken user input.

Assignees

• Apple Inc

Inventors

• Piernot Philippe P
• Binder Justin G

Classifications

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L15/1822

  Parsing for meaning understanding · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title