Device arbitration for listening devices

Patent metadata

Field	Value
Publication number	US-2016155443-A1
Application number	US-201514676647-A
Country	US
Kind code	A1
Filing date	Apr 1, 2015
Priority date	Nov 28, 2014
Publication date	Jun 2, 2016
Grant date	—

Abstract

An electronic device in a topology of interconnected electronic devices can listen for a wake phrase and voice commands. The device can control when and how it responds so that a single device responds to voice commands. Per-task device preferences can be stored for a user. If a preferred device is not available, the task can still be performed on a device that has appropriate capabilities. Machine learning can determine a user's preferences. Power conservation and effective user interaction can result.

First claim

What is claimed is:

1 . A method of controlling which electronic device out of topology of interconnected electronic devices responds to a wake phrase, the method comprising:
in an electronic device of the topology of interconnected electronic devices configured to listen for voice commands, receiving a wake phrase from a microphone of the electronic device; recognizing the wake phrase; and responsive to recognizing the wake phrase, waking the electronic device and controlling which one electronic device out of the topology of interconnected electronic devices plays an audio prompt that indicates an active listening state in response to the wake phrase, wherein the controlling invokes response arbitration rules selecting a single electronic device out of the topology of interconnected electronic devices according to a user preference indicating a primary device designation for the interconnected electronic devices or recorded activity detected by one or more hardware sensors of the electronic device.

2 . The method of claim 1 wherein:
the response arbitration rules specify rules of succession for devices when a primary device is not available.

3 . The method of claim 1 wherein:
the recorded activity indicates physical activity near or at the electronic device; and
the controlling chooses an electronic device having most recent physical activity.

4 . The method of claim 3 wherein:
the electronic device having most recent physical activity is chosen in response to determining that a primary device is not available.

5 . The method of claim 1 wherein:
the controlling action chooses an electronic device designated as a primary device by the user preference.

6 . The method of claim 1 further comprising:
determining whether the wake phrase was spoken by an authorized user;
the wake phrase is not recognized when not spoken by an authorized user.

7 . The method of claim 1 wherein:
the recognizing is performed by an auxiliary processor of the electronic device while the electronic device is in a standby state.

8 . The method of claim 7 wherein:
the controlling is performed by a main processor of the electronic device after the electronic device has transitioned out of the standby state; and
the electronic device consumes less power when in the standby state.

9 . The method of claim 8 wherein:
waking the electronic device activates the main processor from an inactive state.

10 . The method of claim 1 further comprising:
receiving a voice command with the microphone of the electronic device;
recognizing a task in the voice command received with the microphone; and
responsive to recognizing the task, controlling which one electronic device out of the topology of interconnected electronic devices performs the recognized task, wherein the controlling invokes task arbitration rules selecting a single electronic device out of the topology of interconnected electronic devices according to user device preference or stored capabilities of the interconnected electronic devices.

11 . The method of claim 10 wherein:
the task arbitration rules specify rules of succession for devices when a preferred device for the task is not available.

12 . The method of claim 10 further comprising:
recognizing repeated, consistent usage of a particular device for a particular task;
setting the user preference to indicate the particular device as associated with the particular task before listening;
wherein the recognized task comprises the particular task, and controlling which one electronic device performs the task comprises choosing the particular device for the particular task based on the user preference.

13 . The method of claim 10 further comprising:
determining via the stored capabilities of the interconnected electronic devices that the recognized task is not performable at the electronic device; and
selecting an other electronic device to perform the recognized task.

14 . The method of claim 13 further comprising:
handing the recognized task off to an other electronic device.

15 . The method of claim 13 further comprising:
sending a wake command to the other electronic device.

16 . The method of claim 13 wherein:
the selected single electronic device did not hear the voice command.

17 . An electronic device configured to recognize a wake phrase when in a low-power standby state in an environment comprising a topology of interconnected electronic devices, the electronic device comprising:
a plurality of hardware sensors comprising a microphone; a speaker; a stored preference indicating a primary device designation for the topology of interconnected electronic devices; and a stored device listening state, wherein the stored device listening state transitions from the low-power standby state upon recognition of the wake phrase as received by the microphone, and wherein the electronic device is configured to control which one electronic device out of the topology of interconnected electronic devices transitions to an active listening state and plays an audio prompt that indicates the active listening state in response to the wake phrase according to the stored preference indicating the primary device designation for the interconnected electronic devices or recorded activity detected by one or more of the hardware sensors of the electronic device.

18 . The electronic device of claim 17 wherein the electronic device further comprises:
an auxiliary voice recognition subsystem configured to recognize the wake phrase and transition the electronic device out of the low-power standby state; and
a main voice recognition subsystem configured to control which one electronic device out of the topology of interconnected electronic devices transitions to an active listening state.

19 . The electronic device of claim 17 further comprising:
a voice recognizer configured to recognize a voice command received by the microphone of the electronic device, wherein the voice command comprises a task;
a task arbiter configured to control which one electronic device out of the topology of interconnected electronic devices performs the task according to user preference or capabilities of the interconnected electronic devices.

20 . One or more machine-readable media comprising machine-executable instructions causing an electronic device to perform a method controlling which electronic device out of topology of interconnected electronic devices responds to a wake phrase, the method comprising:
receiving a wake phrase of a virtual persona from a microphone of the electronic device; recognizing the wake phrase of the virtual persona in the electronic device; and responsive to recognizing the wake phrase of the virtual persona, waking the electronic device and controlling which one electronic device out of the topology of interconnected electronic devices plays an audio prompt that indicates a listening state in response to the wake phrase, wherein the controlling invokes arbitration rules selecting a single device out of the topology of interconnected electronic devices when a user preference indicating a primary device designation for the interconnected electronic devices indicates a primary device that is not available, based on recorded activity detected by one or more hardware sensors of the electronic device indicating that the single device is currently being used; receiving a command phrase from the microphone of the electronic device; recognizing a

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Khan Yaser
• Uzelac Aleksandar
• Hwang Daniel J
• Paolantonio Sergio
• Kam Jenny
• Kannan Vishwac Sena
• Mooney Ii Dennis James
• Brush Alice Jane Bernheim

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F1/3209

  Monitoring remote activity, e.g. over telephone lines or network connections · CPC title

• G06F3/147

  using display panels · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title