Adaptive beamforming to create reference channels

Patent metadata

Field	Value
Publication number	US-9747920-B2
Application number	US-201514973274-A
Country	US
Kind code	B2
Filing date	Dec 17, 2015
Priority date	Dec 17, 2015
Publication date	Aug 29, 2017
Grant date	Aug 29, 2017

Abstract

An echo cancellation system that performs audio beamforming to separate audio input into multiple directions and determines a target signal and a reference signal from the multiple directions. For example, the system may detect a strong signal associated with a speaker and select the strong signal as a reference signal, selecting another direction as a target signal. The system may determine a speech position and may select the speech position as a target signal and an opposite direction as a reference signal. The system may create pairwise combinations of opposite directions, with an individual direction being selected as a target signal and a reference signal. The system may select a fixed beamformer output for the target signal and an adaptive beamformer output for the reference signal, or vice versa. The system may remove the reference signal (e.g., audio output by the loudspeaker) to isolate speech included in the target signal.

First claim

What is claimed is:

1. A computer-implemented method for cancelling an echo from an audio signal to isolate received speech, the method comprising:
sending a first output audio signal to a first wireless speaker;
receiving a first input audio signal from a first microphone of a microphone array, the first input audio signal including a first representation of audible sound output by the first wireless speaker and a first representation of speech input;
receiving a second input audio signal from a second microphone of the microphone array, the second input audio signal including a second representation of the audible sound output by the first wireless speaker and a second representation of the speech input;
performing first audio beamforming to determine a first portion of combined input audio data comprising a first portion of the first input audio signal corresponding to a first direction and a first portion of the second input audio signal corresponding to the first direction;
performing second audio beamforming to determine a second portion of the combined input audio data comprising a second portion of the first input audio signal corresponding to a second direction and a second portion of the second input audio signal corresponding to the second direction;
selecting at least the first portion as a target signal on which to perform echo cancellation;
selecting at least the second portion as a reference signal to remove from the target signal;
removing the reference signal from the target signal to generate a second output audio signal including a third representation of the speech input;
performing speech recognition processing on the second output audio signal to determine a command; and
executing the command.

2. The computer-implemented method of claim 1 , further comprising:
determining that the second portion corresponds to a highest amplitude representation of the audible sound output of a plurality of portions;
determining that an amplitude of the second portion is above a threshold;
associating the second portion with the first wireless speaker;
selecting the second portion as the reference signal; and
selecting remaining portions of the plurality of portions as the target signal.

3. The computer-implemented method of claim 1 , further comprising:
determining that the speech input is associated with the first direction;
selecting the first portion as the target signal; and
selecting at least the second portion as the reference signal.

4. The computer-implemented method of claim 1 , further comprising:
determining that the second portion corresponds to a highest amplitude representation of the audible sound output of a plurality of portions;
determining that an amplitude of the second portion is below a threshold;
selecting the first portion as the target signal;
determining that the second direction is opposite the first direction;
selecting the second portion as the reference signal;
selecting the second portion as a second target signal;
selecting the first portion as a second reference signal;
removing the reference signal from the target signal to generate the second output audio signal; and
removing the second reference signal from the second target signal to generate a third output audio signal.

5. A computer-implemented method, comprising:
receiving first input audio data from a first microphone of a microphone array, the first input audio data including a first representation of sound output by a first wireless speaker and a first representation of speech input;
receiving second input audio data from a second microphone of the microphone array, the second input audio data including a second representation of the audible sound output by the first wireless speaker and a second representation of the speech input;
performing first audio beamforming to determine a first portion of combined input audio data comprising a first portion of the first input audio signal corresponding to a first direction and a first portion of the second input audio signal corresponding to the first direction;
performing second audio beamforming to determine a second portion of the combined input audio data comprising a second portion of the first input audio signal corresponding to a second direction and a second portion of the second input audio signal corresponding to the second direction;
selecting at least the first portion as a target signal;
selecting at least the second portion as a reference signal; and
removing the reference signal from the target signal to generate first output audio data including a third representation of the speech input.

6. The computer-implemented method of claim 5 , further comprising:
sending second output audio data to the first wireless speaker;
determining that the second portion corresponds to a highest amplitude of a plurality of portions;
determining that an amplitude of the second portion is above a threshold; and
associating the second portion with the first wireless speaker.

7. The computer-implemented method of claim 5 , further comprising:
determining that an amplitude associated with the second portion is above a threshold;
determining that a highest amplitude associated with remaining portions of a plurality of portions is below the threshold;
selecting the second portion as the reference signal; and
selecting the remaining portions as the target signal.

8. The computer-implemented method of claim 5 , further comprising:
determining that a first amplitude associated with the second portion is above a threshold;
determining that a second amplitude associated with a third portion of a plurality of portions is above the threshold;
selecting the second portion as the reference signal;
selecting the third portion as a second reference signal;
selecting at least the first portion as the target signal; and
removing the reference signal and the second reference signal from the target signal to generate the first output audio data.

9. The computer-implemented method of claim 5 , further comprising:
determining that a first amplitude associated with the first portion is above a threshold;
determining that a second amplitude associated with the second portion is above the threshold;
determining that the speech input is associated with the first direction;
selecting the first portion as the target signal; and
selecting the second portion as the reference signal.

10. The computer-implemented method of claim 5 , further comprising:
determining that the speech input is associated with the first direction selecting the first portion as the target signal;
determining that the second direction is opposite the first direction; and
selecting at least the second portion as the reference signal.

11. The computer-implemented method of claim 5 , further comprising:
determining that the second portion corresponds to a highest amplitude of a plurality of portions;
determining that an amplitude of the second portion is below a threshold;
selecting the first portion as the target signal;
determining that the second direction is opposite the first direction;
selecting the second portion as the reference signal;
selecting the second portion as a second target signal;
selecting the first portion as a second reference signal; and
removing the second reference signal from the second target signal to generate second output audio data including a fourth representation of the speech input.

12. The computer-implemented method of claim 5 , further comprising:
performing the first audio beamforming to determine the first portion using a fixed beamforming technique;
perf

Assignees

• Amazon Tech Inc

Inventors

• Ayrapetian Robert
• Hilmes Philip Ryan

Classifications

• G10L21/0216 Primary

  characterised by the method used for estimating noise · CPC title

• H04R5/04 Primary

  Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} · CPC title

• G10L2021/02082

  the noise being echo, reverberation of the speech · CPC title

• G10L2021/02166

  Microphone arrays; Beamforming · CPC title

• H04R2203/12

  Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays · CPC title