Processing of audio signals from multiple microphones

Patent metadata

Field	Value
Publication number	US-12244994-B2
Application number	US-202217814660-A
Country	US
Kind code	B2
Filing date	Jul 25, 2022
Priority date	Jul 27, 2021
Publication date	Mar 4, 2025
Grant date	Mar 4, 2025

Abstract

A first device includes a memory configured to store instructions and one or more processors configured to receive audio signals from multiple microphones. The one or more processors are configured to process the audio signals to generate direction-of-arrival information corresponding to one or more sources of sound represented in one or more of the audio signals. The one or more processors are also configured to and send, to a second device, data based on the direction-of-arrival information and a class or embedding associated with the direction-of-arrival information.

First claim

What is claimed is:

1. A first device comprising:
a memory configured to store data; and
one or more processors, coupled to the memory, wherein the one or more processors are configured to:
receive audio signals from multiple microphones;
process the audio signals to generate direction-of-arrival information corresponding to one or more sources of sound represented in one or more of the audio signals;
send, to a second device, the data that is based on the direction-of-arrival information and a class or embedding associated with the direction-of-arrival information; and
selectively send, based on an amount of available power at the first device, a representation of the audio signals to the second device.

2. The first device of claim 1 , wherein the one or more processors are further configured to process signal data corresponding to the audio signals to determine the class or embedding.

3. The first device of claim 2 , wherein the one or more processors are further configured to perform a beamforming operation on the audio signals to generate the signal data.

4. The first device of claim 2 , wherein the one or more processors are further configured to process the signal data at one or more classifiers to determine the class from among multiple classes supported by the one or more classifiers, for a sound represented in one or more of the audio signals and associated with an audio event, and wherein the class is sent to the second device.

5. The first device of claim 2 , wherein the one or more processors are further configured to process the signal data at one or more encoders to generate the embedding, the embedding corresponding to a sound represented in one or more of the audio signals and associated with an audio event, and wherein the embedding is sent to the second device.

6. The first device of claim 1 , wherein one or more processors are further configured to process image data at one or more encoders to generate the embedding, the embedding corresponding to an object represented in the image data and associated with an audio event, and wherein the embedding is sent to the second device.

7. The first device of claim 6 , further comprising one or more cameras configured to generate the image data.

8. The first device of claim 1 , wherein:
the class corresponds to a category for a particular sound represented in the audio signals and associated with a particular audio event; and
the embedding includes a signature or information that corresponds to the particular sound or the particular audio event and is configured to enable detection, via processing of other audio signals, of the particular sound or the particular audio event in the other audio signals.

9. The first device of claim 1 , wherein the one or more processors are further configured to:
perform spatial processing on the audio signals based on the direction-of-arrival information to generate one or more beamformed audio signals, wherein the representation of the audio signals corresponds to the one or more beamformed audio signals.

10. The first device of claim 1 , wherein the memory and the one or more processors are integrated into a headset device, and wherein the second device corresponds to a mobile phone.

11. The first device of claim 1 , further comprising a modem, wherein the data is sent to the second device via the modem.

12. The first device of claim 1 , wherein the representation of the audio signals corresponds to the audio signals, one or more beamformed audio signals, or reduced versions of the audio signals.

13. The first device of claim 1 , wherein the one or more processors are further configured to generate a user interface output indicative of at least one of an environmental event or an acoustic event.

14. The first device of claim 1 , wherein the one or more processors are further configured to receive, from the second device, data indicative of an acoustic event that is associated with a sound represented in one or more of the audio signals.

15. The first device of claim 1 , wherein the one or more processors are further configured to:
receive, from the second device, directional information associated with the audio signals; and
perform an audio zoom operation based on the directional information.

16. The first device of claim 1 , wherein the one or more processors are integrated in a vehicle.

17. The first device of claim 1 , wherein the data based on the direction-of-arrival information includes a report indicating at least one detected event and a direction of the detected event.

18. The first device of claim 1 , further comprising the multiple microphones.

19. The first device of claim 1 , further comprising at least one speaker configured to output a sound associated with at least one of the audio signals.

20. A method of processing audio, the method comprising:
receiving, at one or more processors of a first device, audio signals from multiple microphones;
processing the audio signals to generate direction-of-arrival information corresponding to one or more sources of sound represented in one or more of the audio signals;
sending, to a second device, data based on the direction-of-arrival information and a class or embedding associated with the direction-of-arrival information; and
selectively sending, based on an amount of available power at the first device, a representation of the audio signals to the second device.

21. The first device of claim 1 , wherein, to selectively send the representation of the audio signals, the one or more processors are further configured to:
compare the amount of available power to a power threshold; and
send the representation of the audio signals to the second device based on the amount of available power being greater than or equal to the power threshold.

22. The first device of claim 1 , wherein, to selectively send the representation of the audio signals, the one or more processors are further configured to:
compare the amount of available power to a power threshold;
determine whether beamformed audio signals are available based on the amount of available power being less than the power threshold; and
determine not to send the representation of the audio signals to the second device based on the beamformed audio signals being unavailable.

23. The first device of claim 1 , wherein, to selectively send the representation of the audio signals, the one or more processors are further configured to:
compare the amount of available power to a power threshold;
determine whether beamformed audio signals are available based on the amount of available power being less than the power threshold; and
send the representation of the audio signals to the second device based on the beamformed audio signals being available, wherein the representation of the audio signals corresponds to the beamformed audio signals.

24. The method of claim 20 , further comprising:
processing signal data corresponding to the audio signals to determine the class or embedding; and
performing a beamforming operation on the audio signals to generate the signal data.

25. The method of claim 20 , further comprising:
processing signal data corresponding to the audio signals to determine the class or embedding, wherein the signal data is processed at one or more classifiers to determine the class from among multiple classes supported by the one or more classifiers, for a sound represented in on

Assignees

• Qualcomm Inc

Inventors

• Visser Erik
• Saki Fatemeh
• Guo Yinyi
• Kim Lae-Hoon
• Alves Rogerio Guedes
• Pessentheiner Hannes

Classifications

• H04R2430/23

  Direction finding using a sum-delay beam-former · CPC title

• H04R2410/01

  Noise reduction using microphones having different directional characteristics · CPC title

• H04R1/406 Primary

  microphones · CPC title

• H04R1/1083

  Reduction of ambient noise (active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06) · CPC title

• H04R5/027

  Spatial or constructional arrangements of microphones, e.g. in dummy heads · CPC title