Echo latency estimation

Patent metadata

Field	Value
Publication number	US-9947338-B1
Application number	US-201715708772-A
Country	US
Kind code	B1
Filing date	Sep 19, 2017
Priority date	Sep 19, 2017
Publication date	Apr 17, 2018
Grant date	Apr 17, 2018

Abstract

A device that determines an echo latency estimate by combining reference signals. The device may determine the echo latency corresponding to an amount of time between reference signals being sent to transmitters and input data corresponding to the reference signals being received. The device may generate a combined reference signal by adding (or filtering) each of the reference signals. The device may then compare the combined reference signal to input audio data received from a microphone or receiving device. The device may detect a highest peak, determine if there are any earlier significant peaks and estimate the echo latency based on the earliest significant peak. This technique is not limited to audio data and may be used for signal matching using any system that includes multiple transmitters and receivers (e.g., Radar, Sonar, etc.).

First claim

What is claimed is:

1. A computer-implemented method comprising:
sending a first reference signal to a first loudspeaker during a first time period, the first reference signal corresponding to a first channel of a song;
sending a second reference signal to a second loudspeaker during the first time period, the second reference signal corresponding to a second channel of the song;
generating a combined reference audio signal using the first reference signal and the second reference signal;
receiving input audio data, the input audio data generated by at least one microphone, the input audio data including a first representation of first audio generated by the first loudspeaker and a second representation of second audio generated by the second loudspeaker;
determining cross correlation data corresponding to a cross correlation between the input audio data and the combined reference signal;
determining a first peak represented in the cross correlation data, the first peak corresponding to a second time period;
determining a second peak represented in the cross correlation data, the second peak corresponding to a third time period;
determining that the second time period is earlier than the third time period;
determining an echo latency estimate by determining a difference between the second time period and the first time period, the echo latency estimate indicating an amount of time between sending a reference signal and capturing audio corresponding to the reference signal;
determining, using the echo latency estimate, at least one of a step size control value, a tail length value or a reference delay value; and
performing acoustic echo cancellation using at least one of the step size control value, the tail length value or the reference delay value.

2. The computer-implemented method of claim 1 , wherein generating the combined reference signal further comprises:
determining a first impulse response associated with the first loudspeaker, the first impulse response corresponding to a first environment in which the first loudspeaker is located;
determining first filter coefficient values modeling the first impulse response;
generating a first filtered reference signal using the first filter coefficient values and the first reference signal;
determining a second impulse response associated with the second loudspeaker, the second impulse response corresponding to a second environment in which the second loudspeaker is located;
determining second filter coefficient values modeling the second impulse response;
generating a second filtered reference signal using the second filter coefficient values and the second reference signal; and
generating the combined reference signal by combining the first filtered reference signal and the second filtered reference signal.

3. The computer-implemented method of claim 1 , further comprising:
determining that a first value is a highest value in the cross correlation data, the first value corresponding to the second peak;
determining a second value that is a highest value associated with the first peak;
determining a ratio between the first value and the second value;
determining that the ratio is above a threshold value, the threshold value indicating whether the first peak is high enough to be used to determine the echo latency estimate; and
determining the echo latency estimate using the second time period associated with the second value.

4. The computer-implemented method of claim 1 , further comprising:
determining a first portion of the first reference signal;
determining a second portion of the first reference signal, the second portion overlapping the first portion for a duration of time;
determining second cross correlation data corresponding to a second cross correlation between the first portion and the second portion;
determining that the second cross correlation data only includes a single peak; and
sending the first reference signal to the first loudspeaker.

5. A computer-implemented method comprising:
sending first audio data that corresponds to a first loudspeaker during a first time period;
sending second audio data that corresponds to a second loudspeaker during the first time period;
generating third audio data based on the first audio data and the second audio data;
receiving input audio data, the input audio data generated by at least one microphone;
determining cross correlation data corresponding to a cross correlation between the input audio data and the third audio data;
determining a first peak represented in the cross correlation data, the first peak corresponding to a second time period; and
determining an estimated latency based on a difference between the second time period and the first time period, the estimated latency corresponding to a delay between sending the first audio data or the second audio data and the at least one microphone capturing audio corresponding to the first audio data or the second audio data.

6. The computer-implemented method of claim 5 , wherein generating the third audio data further comprises:
determining first characteristics associated with the first loudspeaker;
determining first filter coefficient values corresponding to the first characteristics;
generating first filtered audio data using the first filter coefficient values and the first audio data;
determining second characteristics associated with the second loudspeaker;
determining second filter coefficient values corresponding to the second characteristics;
generating second filtered audio data using the second filter coefficient values and the second audio data; and
generating the third audio data by combining the first filtered audio data and the second filtered audio data.

7. The computer-implemented method of claim 5 , further comprising:
determining a first value that is a highest value in the cross correlation data, the first value corresponding to the first peak;
determining a second peak represented in the cross correlation data, the second peak corresponding to a third time period prior to the second time period;
determining a second value that is a highest value associated with the second peak;
determining a ratio between the first value and the second value;
determining that the ratio is below a threshold value; and
determining the estimated latency based on the second time period associated with the first value.

8. The computer-implemented method of claim 5 , further comprising:
determining that a first value is a highest value in the cross correlation data;
determining a second peak represented in the cross correlation data that includes the first value, the second peak corresponding to a third time period;
determining the first peak represented in the cross correlation data, the first peak corresponding to the second time period, the second time period being prior to the third time period;
determining a second value that is a highest value associated with the first peak;
determining a ratio between the first value and the second value;
determining that the ratio is above a threshold value; and
determining the estimated latency based on the second time period associated with the second value.

9. The computer-implemented method of claim 5 , further comprising:
determining a first number of loudspeakers to which audio data is sent during the first time period;
determining a second number of peaks in the cross correlation data, the second number equal to the first number;
determining, from the second number of peaks, a highest peak in the cross correlation data; and
selecting the highest peak as the first peak.

10. The computer-implemented method of claim 5 , further

Assignees

• Amazon Tech Inc

Inventors

• Koteshwara Krishna Kamath
• Kristjansson Trausti Thor

Classifications

• H04R5/02

  Spatial or constructional arrangements of loudspeakers · CPC title

• G10L2021/02082

  the noise being echo, reverberation of the speech · CPC title

• G10L21/0232 Primary

  Processing in the frequency domain · CPC title

• H04S7/301

  Automatic calibration of stereophonic sound system, e.g. with test microphone · CPC title

• H04S3/008 Primary

  in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02) · CPC title