Multi-device audio capture

Patent metadata

Field	Value
Publication number	US-10477294-B1
Application number	US-201815883663-A
Country	US
Kind code	B1
Filing date	Jan 30, 2018
Priority date	Jan 30, 2018
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

A system and method includes a first device (e.g., a primary or primary earbud) that detects first audio corresponding to an utterance and a second device (e.g., a secondary or secondary earbud) that detects second audio corresponding to the utterance. The first device determines first audio data corresponding to the first audio and the second device determines second audio data corresponding to the second audio. The first device determines a first quality metric corresponding to the first audio data, and the second device determines a second quality metric corresponding to the second audio data. The second device transmits the second quality metric to first device, which compares it to the first quality metric. If the first device determines the second quality metric is better, it transmits the second audio data to a third device (e.g., a smartphone) and enters a power-saving state.

First claim

What is claimed is:

1. A computer-implemented method comprising:
detecting first audio using a primary wireless earbud, wherein the primary wireless earbud:
comprises a first set of microphones,
is wirelessly connected to a secondary wireless earbud over a first wireless communication connection, and
is wirelessly connected to a mobile device over a second wireless communication connection;

determining, by the primary wireless earbud, first audio data corresponding to the first audio;
determining, by the primary wireless earbud, that the first audio data corresponds to speech;
determining, by the primary wireless earbud, a first signal-to-noise ratio (SNR) corresponding to the first audio data;
receiving, by the primary wireless earbud from the secondary wireless earbud:
first data corresponding to the secondary wireless earbud detecting speech in second audio data corresponding to second audio detected by the secondary wireless earbud, and
a second SNR corresponding to the second audio data;

determining that the second SNR is higher than the first SNR;
receiving, by the primary wireless earbud from the secondary wireless earbud, at least a portion of the second audio data; and
sending, by the primary wireless earbud, at least the portion of the second audio data to the mobile device.

2. The method of claim 1 , further comprising, prior to sending the second audio data, at least one of:
determining, by the primary wireless earbud, that the first audio data includes a wakeword; or
determining, by the secondary wireless earbud, that the second audio data includes the wakeword.

3. A computer-implemented method comprising:
detecting, by a first earbud of a pair of earbuds, first audio corresponding to a first utterance;
determining, by the first earbud, first audio data corresponding to the first audio;
determining, by the first earbud, a first quality metric corresponding to the first audio data;
receiving, by the first earbud and from a second earbud of the pair of earbuds, a second quality metric corresponding to second audio data corresponding to the first utterance;
determining that the second quality metric corresponds to higher audio quality than the first quality metric;
receiving, by the first earbud and from the second earbud, at least a portion of the second audio data;
sending, by the first earbud and to a user device, output audio data comprising at least the portion of the second audio data; and
causing at least a portion of the first earbud to enter a power-saving state.

4. The method of claim 3 , further comprising:
receiving, by the first earbud from the second earbud, first data corresponding to the second earbud detecting second audio corresponding to the second audio data; and
sending, from the first earbud to the second earbud, a command to send the second audio data.

5. The method of claim 3 , further comprising, prior to sending the output audio data:
processing, by the first earbud, the first audio data to detect a representation of a wakeword; and
receiving, from the second earbud, an indication of detection of the wakeword by the second earbud.

6. The method of claim 3 , further comprising:
determining that the first quality metric is greater than a threshold; and
configuring the output audio data to further comprise at least a portion of the first audio data.

7. The method of claim 3 , further comprising:
determining, using the first earbud, noise data common to the first audio data and the second audio data; and
removing the noise data from the first audio data and the second audio data to determine filtered audio data,
wherein the output audio data comprises the filtered audio data.

8. The method of claim 3 , further comprising:
determining that the first quality metric is lower than a threshold; and
causing at least the portion of the first earbud to enter the power-saving state based at least in part on the first quality metric being lower than the threshold.

9. The method of claim 3 , wherein causing at least the portion of the first earbud to enter the power-saving state further comprises at least one of:
causing a voice-activity detection component to cease analyzing the first audio data; or
causing a wakeword detection component to cease analyzing the first audio data.

10. The method of claim 3 , wherein determining the first audio data further comprises at least one of:
determining, by the first earbud, the first audio data comprises a voice; or
receiving, by the first earbud from the second earbud, first data corresponding to the second earbud determining the second audio data comprises the voice.

11. The method of claim 10 , wherein determining the second quality metric corresponds to higher audio quality than the first quality metric further comprises at least one of:
determining, by the first earbud, that the first audio data comprises a wakeword; or
receiving, by the first earbud from the second earbud, second data corresponding to the second earbud determining the second audio data comprises the wakeword.

12. A system comprising:
at least one processor; and
at least one memory including instructions that, when executed by the at least one processor, cause the system to:
detect, by a first earbud, first audio corresponding to an utterance,
determine, by the first earbud, first audio data corresponding to the first audio;
determine, by the first earbud, a first quality metric corresponding to the first audio data,
receive, by the first earbud and from a second earbud, a second quality metric corresponding to second audio data corresponding to the first utterance,
determine that the second quality metric corresponds to higher audio quality than the first quality metric,
receive, by the first earbud from the second earbud, at least a portion of the second audio data,
send transmit, by the first earbud to a user device, output audio data comprising at least the portion of the second audio data, and
cause at least a portion of the first earbud to enter a power-saving state.

13. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:
receive, by the first earbud from the second earbud, first data corresponding to the second earbud detecting second audio corresponding to the second audio data; and
send, from the first earbud to the second earbud, a command to send the second audio data.

14. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:
process, by the first earbud, the first audio data to detect a representation of a wakeword; and
receive, from the second earbud, an indication of detection of the wakeword by the second earbud.

15. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:
determine that the first quality metric is greater than a threshold; and
configure the output audio data to further comprise at least a portion of the first audio data.

16. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:
determine, using the first earbud, noise data common to the first audio data and the second audio data; and
remove the noise data from the first audio data and the second audio data to determine filtered audio data,
wherein the output audio data comprises the filtered audio data.

Assignees

• Amazon Tech Inc

Inventors

• Jorgovanovic Milos
• Birch-Jensen Hans Edward

Classifications

• G10L25/60

  for measuring the quality of voice signals · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• H04R2460/03

  Aspects of the reduction of energy consumption in hearing devices · CPC title

• H04R1/1041

  Mechanical or electronic switches, or control elements (switches in general H01H) · CPC title

• H04R2201/107

  Monophonic and stereophonic headphones with microphone for two-way hands free communication · CPC title