Methods and systems for automatically equalizing audio output based on room position

Patent metadata

Field	Value
Publication number	US-11005440-B2
Application number	US-201916719847-A
Country	US
Kind code	B2
Filing date	Dec 18, 2019
Priority date	Oct 4, 2017
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

The various implementations described herein include methods, devices, and systems for automatic audio equalization. In one aspect, a method is performed at an electronic device that includes speakers, microphones, processors and memory. The electronic device outputs audio user content from the speakers and automatically equalizes subsequent audio output of the device without user input. The automatic equalization includes: (1) obtaining audio content signals, including receiving outputted audio content at each microphone; (2) determining from the audio content signals phase differences between microphones; (3) obtaining a feature vector based on the phase differences; (4) obtaining a frequency correction from a correction database based on the obtained feature vector; and (5) applying the obtained frequency correction to the subsequent audio output.

First claim

What is claimed is:

1. A method performed at an electronic device located in a room, the electronic device comprising one or more speakers, one or more microphones, one or more processors, and memory, the method comprising:
outputting first audio content from the one or more speakers;
capturing via the one or more microphones audio data representing reflections of the first audio content;
determining an orientation of the electronic device;
based on the captured audio data, determining a frequency response for the room;
querying without user interaction a correction database to obtain an audio frequency correction corresponding to the frequency response for the room and the orientation of the electronic device, wherein the database comprises audio frequency corrections corresponding to different spatial positions and orientations of audio devices in the room; and
applying by the electronic device the obtained frequency correction to subsequent audio output.

2. The method of claim 1 , further comprising:
prior to the outputting, generating the correction database, wherein the generating includes:
positioning a speaker device at a particular position and orientation within a structure;
outputting via the speaker device training audio;
receiving the outputted training audio at two or more first microphones;
generating a reference frequency correction based on the outputted training audio; and
adding the reference frequency correction to the correction database.

3. The method of claim 1 , wherein the one or more microphones comprise a plurality of microphones, the method further comprising:
determining a plurality of phase differences of the captured audio data between different microphones of the plurality of microphones.

4. The method of claim 3 , further comprising:
obtaining a feature vector based on the plurality of phase differences;
wherein determining the frequency response for the room includes obtaining the feature vector.

5. The method of claim 4 , further comprising assigning a plurality of weights to the plurality of phase differences such that each phase difference of the plurality of phase differences is assigned a corresponding weight; and
wherein the feature vector is based on the weighted plurality of phase differences.

6. The method of claim 5 , wherein the plurality of weights is based on a signal-to-noise ratio for the captured audio data at each microphone of the plurality of microphones.

7. The method of claim 5 , wherein the plurality of weights is based on relative positioning of the plurality of microphones.

8. The method of claim 5 , wherein obtaining the feature vector based on the plurality of phase differences comprises applying a fast Fourier transform (FFT) to the plurality of phase differences.

9. The method of claim 1 , wherein the first audio content includes:
audio content that is selected by a user of the electronic device;
a test signal; and/or
training audio.

10. The method of claim 1 , wherein applying the obtained frequency correction to subsequent audio output comprises adjusting a gain for a particular range of frequencies.

11. An electronic device located in a room, comprising:
one or more speakers;
one or more microphones;
one or more processors; and
memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
outputting first audio content from the one or more speakers;
capturing via the one or more microphones audio data representing reflections of the first audio content;
determining an orientation of the electronic device;
based on the captured audio data, determining a frequency response for the room;
querying without user interaction a correction database to obtain an audio frequency correction corresponding to the frequency response for the room and the orientation of the electronic device, wherein the database comprises audio frequency corrections corresponding to different spatial positions and orientations of audio devices in the room; and
applying by the electronic device the obtained frequency correction to subsequent audio output.

12. The electronic device of claim 11 , wherein the one or more microphones comprise a plurality of microphones, the one or more programs further including instructions for:
determining a plurality of phase differences of the captured audio data between different microphones of the plurality of microphones.

13. The electronic device of claim 12 , the one or more programs further including instructions for:
obtaining a feature vector based on the plurality of phase differences; and
prior to obtaining the feature vector, determining that the outputted first audio content includes audio having a frequency below a transition frequency for the room.

14. The electronic device of claim 13 , the one or more programs further including instructions for:
prior to obtaining the feature vector, determining that the outputted first audio content has an acoustic energy that meets one or more energy criteria for a particular range of frequencies.

15. The electronic device of claim 13 , the one or more programs further including instructions for:
prior to obtaining the feature vector, determining that the outputted first audio content has an audio coherence that meets one or more signal-to-noise criteria.

16. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device located in a room, wherein the electronic device comprises one or more speakers, one or more microphones, one or more processors, and memory, cause the electronic device to perform operations comprising:
outputting first audio content from the one or more speakers;
capturing via the one or more microphones audio data representing reflections of the first audio content;
determining an orientation of the electronic device;
based on the captured audio data, determining a frequency response for the room;
querying without user interaction a correction database to obtain an audio frequency correction corresponding to the frequency response for the room and the orientation of the electronic device, wherein the database comprises audio frequency corrections corresponding to different spatial positions and orientations of audio devices in the room; and
applying by the electronic device the obtained frequency correction to subsequent audio output.

17. The non-transitory computer-readable storage medium of claim 16 , the operations further comprising:
prior to the outputting, generating the correction database, wherein the generating includes:
positioning a speaker device at a particular position and orientation within a structure;
outputting via the speaker device training audio;
receiving the outputted training audio at two or more first microphones;
generating a reference frequency correction based on the outputted training audio; and
adding the reference frequency correction to the correction database.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more microphones comprise a plurality of microphones, the operations further comprising:
determining a plurality of phase differences of the captured audio data between different microphones of the plurality of microphones.

19. The non-transitory computer-readable storage medium of claim 16 , wherein the fir

Assignees

• Google Llc

Inventors

• Shaya Benjamin Louis
• Larsen Rasmus
• Lyon Richard F
• Smedegaard Michael

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title