Method and apparatus for audio capture using beamforming

Patent metadata

Field	Value
Publication number	US-10771894-B2
Application number	US-201716473370-A
Country	US
Kind code	B2
Filing date	Dec 28, 2017
Priority date	Jan 3, 2017
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

An apparatus for capturing audio comprises a first beamformer (305) coupled to a microphone array (301) and arranged to generate a first beamformed audio output. A plurality of constrained beamformers (309, 311) each generates a constrained beamformed audio output. A first adapter (307) adapts beamform parameters of the first beamformer (305) and a second adapter (313) adapts constrained beamform parameters for the plurality of constrained beamformers (309, 311). A difference processor (317) determines a difference measure for the constrained beamformers (309, 311) where the difference measure is indicative of the difference between beams formed by the first beamformer (305) and the constrained beamformers (309, 311). The second adapter (313) is arranged to adapt constrained beamform parameters with the constraint that beamform parameters are adapted only for constrained beamformers of the plurality of constrained beamformers (309, 311) for which a difference measure has been determined that meets a similarity criterion.

First claim

The invention claimed is:

1. An apparatus for capturing audio, the apparatus comprising:
a microphone array;
a first beamformer, the beamformer coupled to the microphone array, wherein the beamformer is arranged to generate a first beamformed audio output;
a plurality of constrained beamformers, the plurality of constrained beamformers coupled to the microphone array, wherein each of the plurality of constrained beamformers is arranged to generate a constrained beamformed audio output;
a first adapter, wherein the first adaptor is arranged to adapt beamform parameters of the first beamformer;
a second adapter, wherein the second adaptor is arranged to adapt constrained beamform parameters for the plurality of constrained beamformers;
a difference processor circuit, wherein the difference processor circuit is arranged to determine a difference measure for at least one of the plurality of constrained beamformers,
wherein the difference measure is indicative of a difference between beams formed by the first beamformer and the at least one of the plurality of constrained beamformers;
wherein the second adapter is arranged to adapt constrained beamform parameters with a constraint such that constrained beamform parameters are adapted only for constrained beamformers of the plurality of constrained beamformers for which a difference measure has been determined that meets a similarity criterion,
wherein the difference processor circuit is arranged to determine the difference measure for a first constrained beamformer as a difference between a first set of parameters and the constrained set of parameters for the first constrained beamformer.

2. The apparatus of claim 1 further comprising an audio source detector,
wherein the audio source detector is arranged to detect point audio sources in the constrained beamformed audio outputs,
wherein the second adapter is arranged to adapt constrained beamform parameters only for constrained beamformers for which a presence of a point audio source is detected in the constrained beamformed audio output.

3. The apparatus of claim 2 ,
wherein the audio source detector is arranged to detect point audio sources in the first beamformed audio output, wherein the apparatus further comprises a controller circuit is arranged to set constrained beamform parameters for a first constrained beamformer in response to beamform parameters of the first beamformer if a point audio source is detected in the first beamformed audio output but not in any constrained beamformed audio outputs.

4. The apparatus of claim 3 ,
wherein the controller circuit is arranged to set the constrained beamform parameters for the first constrained beamformer in response to the beamform parameters of the first beamformer,
wherein the controller circuit is arranged to set the constrained beamform parameters only if a difference measure for the first constrained beamformer exceeds a threshold.

5. The apparatus of claim 2 ,
wherein the audio source detector is arranged to detect audio sources in the first beamformed audio output,
wherein the apparatus further comprises a controller circuit arranged to set constrained beamform parameters for a first constrained beamformer in response to the beamform parameters of the first beamformer,
wherein the controller circuit is arranged to set the constrained beamform parameters if a point audio source is detected in the first beamformed audio output and in a second beamformed audio output from the first constrained beamformer and a difference measure has been determined for the first constrained beamformer which exceeds a threshold.

6. The apparatus of claim 5 ,
wherein the plurality of constrained beamformers is an active subset of the constrained beamformers, wherein the active subset of constrained beamformers is selected from a pool of constrained beamformers,
wherein the controller circuit is arranged to increase a number of active constrained beamformers to include the first constrained beamformer by initializing a constrained beamformer from the pool of constrained beamformers using the beamform parameters of the first beamformer.

7. The apparatus of claim 1 , wherein the second adapter is arranged to only adapt the constrained beamform parameters for a first constrained beamformer if a criterion is met comprising at least one requirement selected from the group of:
a requirement that a level of the second beamformed audio output from the first constrained beamformer is higher than for any other second beamformed audio output,
a requirement that a level of a point audio source in the second beamformed audio output from the first constrained beamformer is higher than any point audio source in any other second beamformed audio output,
a requirement that a signal to noise ratio for the second beamformed audio output from the first constrained beamformer exceeds a threshold, and
a requirement that the second beamformed audio output from the first constrained beamformer comprises a speech component.

8. The apparatus of claim 1 , wherein an adaptation rate for the first beamformer is higher than for the plurality of constrained beamformers.

9. The apparatus of claim 1 wherein the first beamformer and the plurality of constrained beamformers are filter-and-combine beamformers.

10. The apparatus of claim 1 ,
wherein the first beamformer is a filter-and-combine beamformer comprising a first plurality of beamform filters,
wherein each of the first plurality of beamform filters has a first adaptive impulse responses,
wherein a second beamformer is a constrained beamformer of the plurality of constrained beamformers,
wherein the second beamformer is a filter-and-combine beamformer comprising a second plurality of beamform filters,
wherein each of the second plurality of beamform filters has having a second adaptive impulse response,
wherein the difference processor circuit is arranged to determine the difference measure between beams of the first beamformer and the second beamformer in response to a comparison of the first adaptive impulse responses to the second adaptive impulse responses.

11. The apparatus of claim 1 further comprising:
a noise reference beamformer,
wherein the noise reference beamformer is arranged to generate a beamformed audio output signal and at least one noise reference signal,
wherein the noise reference beamformer is one of the first beamformer and the plurality of constrained beamformers;

a first transformer,
wherein the first transform is arranged to generate a first frequency domain signal from a frequency transform of the beamformed audio output signal,
wherein the first frequency domain signal is represented by time frequency tile values;

a second transformer,
wherein the first transform is arranged to generate a second frequency domain signal from a frequency transform of the at least one noise reference signal,
wherein the second frequency domain signal is represented by time frequency tile values;

the difference processor circuit, the difference processor circuit arranged to generate time frequency tile difference measures, wherein a time frequency tile difference measure for a first frequency is indicative of a difference between a first monotonic function of a norm of a time frequency tile value of the first frequency domain signal for the first frequency and a second monotonic function of a norm of a time frequency tile value of the second frequency domain signal for the first frequency; and
a point audio source estimator,
wherein the point audio source estimator is arranged to generate a point audio source estimate indicative of whether the beamformed audio output signal com

Assignees

• Koninklijke Philips Nv

Inventors

• Janse Cornelis Pieter
• Bloemendal Brian Brand Antonius Johannes
• Kechichian Patrick
• Janssen Rik Jozef Martinus

Classifications

• H04R1/406

  microphones · CPC title

• H04R3/005 Primary

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• H04R2430/20

  Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic (H04R2203/12 takes precedence) · CPC title

• G10L2021/02166

  Microphone arrays; Beamforming · CPC title