Systems, devices, and methods of acoustic echo cancellation based on display orientation

Patent metadata

Field	Value
Publication number	US-11620976-B2
Application number	US-202016897039-A
Country	US
Kind code	B2
Filing date	Jun 9, 2020
Priority date	Jun 9, 2020
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

An electronic device includes a microphone array to capture audio input data, a speaker array to render audio output data for playback; one or more sensors to detect an orientation of the microphone array, acoustic echo cancellation logic, and an interface. The acoustic echo cancellation logic applies acoustic echo cancellation to the audio input data to form echo-cancelled audio input data based on the orientation of the microphone array. The interface transmits the echo-cancelled audio input data over a communications channel as part of an audiovisual communication system.

First claim

What is claimed is:

1. An electronic device comprising:
a display device rotatable with respect to a base;
a microphone array configured to capture audio input data, wherein the microphone array if fixedly attached to the display device;
a speaker array configured to render audio output data for playback, wherein the speaker array is housed in the base;
one or more sensors configured to detect data indicating an orientation of the display device with respect to the base;
acoustic echo cancellation logic configured to apply acoustic echo cancellation to the audio input data to form echo-cancelled audio input data based on the data indicating the orientation of the display device with respect to the base; and
an interface configured to transmit the echo-cancelled audio input data over a communications channel as part of an audiovisual communication system.

2. The electronic device of claim 1 , wherein the acoustic echo cancellation logic is further configured to determine a set of echo cancellation parameters based on the orientation of the microphone array, and wherein to apply the acoustic echo cancellation to the audio input data, the acoustic echo cancellation logic is configured to apply the acoustic echo cancellation based on the determined set of echo cancellation parameters.

3. The electronic device of claim 2 , wherein to apply the acoustic echo cancellation to the audio data, the acoustic echo cancellation logic is configured to:
configure an adaptive filter based on the determined set of echo cancellation parameters; and
apply, to the audio data, the adaptive filter configured based on the determined set of echo cancellation parameters.

4. The electronic device of claim 3 , wherein to configure the adaptive filter, the acoustic echo cancellation logic is configured to control an adaptation or a training of the adaptive filter based on the determined set of echo cancellation parameters.

5. The electronic device of claim 3 , wherein to determine the set of echo cancellation parameters, the acoustic echo cancellation logic is configured to select the set of echo cancellation parameters from a plurality of sets of echo cancellation parameters using a lookup table.

6. The electronic device of claim 3 , wherein to determine the set of echo cancellation parameters, the acoustic echo cancellation logic is configured to apply a finite element solution that determines the set of echo cancellation parameters.

7. The electronic device of claim 3 , wherein to determine the set of echo cancellation parameters, the acoustic echo cancellation logic is configured to apply an artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of echo cancellation parameters to predict the set of echo cancellation parameters.

8. The electronic device of claim 3 , wherein to apply the adaptive filter to the audio data, the acoustic echo cancellation logic is configured to determine signal coherence among signals of the audio data using the adaptive filter.

9. The electronic device of claim 1 , wherein the acoustic echo cancellation logic is further configured to determine a relative positioning between the speaker array and the microphone array based on the data indicating the orientation of the display device detected by the one or more sensors, and wherein to apply the acoustic echo cancellation to the audio data, the acoustic echo cancellation logic is configured to: determine a set of echo cancellation parameters based on the relative positioning between the speaker array and the microphone array; and apply the acoustic echo cancellation based on the determined set of echo cancellation parameters.

10. The electronic device of claim 1 , wherein the one or more sensors includes one or more of an accelerometer, a position encoder, a gyroscope, a motion sensor, camera hardware, an imaging system, a radio frequency probe, an antenna, or a Hall probe.

11. The electronic device of claim 1 , wherein the one or more sensors is further configured to detect configured to detect an orientation of a physical component of the electronic device relative the microphone array, and wherein to apply the acoustic echo cancellation to the audio input data to form the echo-cancelled audio input data based on the orientation of the microphone array, the acoustic echo cancellation logic is configured to process the audio data to form the echo-cancelled audio input data based on the orientation of the physical component relative to the microphone array.

12. A method comprising:
capturing, by a microphone array fixedly attached to a display of an electronic device, audio input data;
rendering, by a speaker array housed in a base of the electronic device, audio output data for playback, wherein the display is rotatable with respect to the base;
detecting, by one or more sensors of the electronic device, data indicating an orientation of the display with respect to the base;
applying, by acoustic echo cancellation logic of the electronic device, based on the data indicating the orientation of the display with respect to the base, acoustic echo cancellation to the audio input data to form echo-cancelled audio input data; and
transmitting, via an interface of the electronic device, the echo-cancelled audio input data over a communications channel.

13. The method of claim 12 , further comprising determining, by the acoustic echo cancellation logic, a set of echo cancellation parameters based on the orientation of the microphone array, wherein applying the acoustic echo cancellation to the audio input data comprises applying, by the acoustic echo cancellation logic the acoustic echo cancellation based on the determined set of echo cancellation parameters.

14. The method of claim 13 , wherein applying the acoustic echo cancellation to the audio data comprises:
configuring, by the acoustic echo cancellation logic, an adaptive filter based on the determined set of echo cancellation parameters; and
applying, by the acoustic echo cancellation logic, to the audio data, the adaptive filter configured based on the determined set of echo cancellation parameters.

15. The method of claim 14 , wherein determining the set of echo cancellation parameters comprises selecting, by the acoustic echo cancellation logic, the set of echo cancellation parameters from a plurality of sets of echo cancellation parameters using a lookup table.

16. The method of claim 14 , wherein determining the set of echo cancellation parameters comprises applying, by the acoustic echo cancellation logic, a finite element solution that determines the set of echo cancellation parameters.

17. The method of claim 14 , wherein determining the set of echo cancellation parameters comprises applying, by the acoustic echo cancellation logic, artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of echo cancellation parameters to predict the set of echo cancellation parameters.

18. The method of claim 14 , wherein applying the adaptive filter to the audio data comprises determining, by the acoustic echo cancellation logic, signal coherence among signals of the audio data using the adaptive filter.

19. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of an electronic device having a display, an interface, sensors, and a speaker array, the one or more programs including instruction

Assignees

• Meta Platforms Tech Llc

Inventors

• Back Tyler
• Bingham Joshua Warren
• Zeng yuhuan
• Ivanov Plamen
• Evans Christopher
• Nilsson Jens
• Asfaw Michael

Classifications

• G10L21/0208

  Noise filtering · CPC title

• H04S7/303

  Tracking of listener position or orientation · CPC title

• H04M3/568 Primary

  audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants (echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30) · CPC title

• G10L2021/02082

  the noise being echo, reverberation of the speech · CPC title

• H04R1/406

  microphones · CPC title