Systems, devices, and methods of manipulating audio data based on microphone orientation

Patent metadata

Field	Value
Publication number	US-11340861-B2
Application number	US-202016897018-A
Country	US
Kind code	B2
Filing date	Jun 9, 2020
Priority date	Jun 9, 2020
Publication date	May 24, 2022
Grant date	May 24, 2022

Abstract

An electronic device includes a microphone array configured to capture audio data, one or more sensors configured to detect an orientation of the microphone array, and digital signal processing (DSP) logic, and an interface. The DSP logic processes, based on the orientation of the microphone array detected by the one or more sensors, the audio data captured by the microphone array to form audio input data. The interface configured to transmit the audio input data over a communications channel to be output by another electronic device.

First claim

What is claimed is:

1. An electronic device comprising:
a display device repositionable relative to a base;
a microphone array configured to capture audio data, the microphone array fixedly attached to the display device;
one or more audio output devices housed in the base;
one or more sensors configured to detect an orientation of the display device relative to the base;
digital signal processing (DSP) logic configured to process, based on the orientation of the display device relative to the base detected by the one or more sensors, the audio data captured by the microphone array to form audio input data;
an interface configured to transmit the audio input data over a communications channel to the one or more audio output devices; and
driver logic that drives the one or more audio output devices based on the orientation of the display device relative to the base.

2. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to drop signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

3. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to constrain a search space to signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

4. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to process the audio data using a set of audio capture parameters corresponding to the orientation of the microphone array.

5. The electronic device of claim 4 , wherein to determine the set of audio capture parameters corresponding to the orientation of the microphone array, the DSP logic is configured to select the set of audio capture parameters corresponding to the orientation of the microphone array from a plurality of sets of audio capture parameters using a lookup table.

6. The electronic device of claim 4 , wherein to determine the set of audio capture parameters corresponding to the orientation of the microphone array, the DSP logic is configured to apply a finite element solution that determines the set of audio capture parameters corresponding to the orientation of the microphone array.

7. The electronic device of claim 4 , wherein to determine the set of audio capture parameters corresponding to the orientation of the microphone array, the DSP logic is configured to apply an artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of audio capture parameter sets to predict the set of audio capture parameters corresponding to the orientation of the microphone array.

8. The electronic device of claim 1 , wherein:
the DSP logic is configured to determine horizontal coordinate angles of the microphone array based on the orientation of the display device, and
to process the audio data to form the audio input data, the DSP logic is configured to process the audio data using the horizontal coordinate angles of the microphone array.

9. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to:
generate a virtual directional microphone associated with one or more individual microphones of the microphone array based on the orientation of the display device array; and
form the audio input data using signals received from the one or more individual microphones that correspond to the virtual directional microphone.

10. The electronic device of claim 1 , wherein the one or more sensors include one or more of an accelerometer, a position encoder, a gyroscope, or a motion sensor.

11. A method comprising:
capturing, by a microphone array, audio data;
detecting, by one or more sensors, an orientation of a display device relative to a base, wherein the microphone array is fixedly attached to the display device, and the display device is repositionable relative to the base;
processing, by digital signal processing (DSP) logic, based on the orientation of the display device relative to the base detected by the one or more sensors, the audio data captured by the microphone array to form audio input data; and
transmitting, by an input/output interface, the audio input data over a communications channel to a speaker array housed in the base; and
driving the speaker array based on the orientation of the display device relative to the base.

12. The method of claim 11 , wherein processing the audio data to form the audio input data comprises dropping, by the DSP logic, signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

13. The method of claim 11 , wherein processing the audio data to form the audio input data comprises constraining, by the DSP logic, a search space to signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

14. The method of claim 11 , wherein processing the audio data to form the audio input data comprises processing, by the DSP logic, the audio data using a set of audio capture parameters corresponding to the orientation of the display device.

15. The method of claim 14 , wherein determining the set of audio capture parameters corresponding to the orientation of the microphone array comprises applying, by the driver logic, an artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of audio capture parameter sets to predict the set of audio capture parameters corresponding to the orientation of the device.

16. The method of claim 14 , wherein determining the set of audio capture parameters corresponding to the orientation of the microphone array comprises selecting, by the driver logic, the set of audio capture parameters corresponding to the orientation of the display device from a plurality of sets of audio capture parameters using a lookup table.

17. The method of claim 14 , wherein determining the set of audio capture parameters corresponding to the orientation of the display device comprises applying, by the driver logic, a finite element solution that determines the set of audio capture parameters corresponding to the orientation of the display device.

18. The method of claim 11 , further comprising:
determining, by the DSP logic, horizontal coordinate angles of the microphone array based on the orientation of the display device,
wherein processing the audio data to form the audio input data comprises processing, by the DSP logic, the audio data using the horizontal coordinate angles of the microphone array.

19. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of an electronic device having a microphone array, an interface, one or more sensors, and a speaker array, the one or more programs including instructions, which when executed by the one or more processors, cause the electronic device to:
capture, using the microphone array, audio data;
detect, using one or more sensors, an orientation of a display device relative to a base, wherein the microphone array is fixedly attached to the display device, and the display device is repositionable relative to the base;
process, based on the orientation of the

Assignees

• Facebook Tech Llc

Inventors

• Back Tyler
• Bingham Joshua Warren
• Zeng yuhuan
• Ivanov Plamen
• Evans Christopher
• Nilsson Jens
• Asfaw Michael

Classifications

• H04N7/15

  Conference systems · CPC title

• H04R2499/15

  Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops · CPC title

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• H04M3/568 Primary

  audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants (echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30) · CPC title

• H04R1/406

  microphones · CPC title