Spatial audio processing apparatus

Patent metadata

Field	Value
Publication number	US-10382849-B2
Application number	US-201615742240-A
Country	US
Kind code	B2
Filing date	Jul 5, 2016
Priority date	Jul 8, 2015
Publication date	Aug 13, 2019
Grant date	Aug 13, 2019

Abstract

Apparatus including: an audio capture application configured to determine separate microphones from a plurality of microphones and identify a sound source direction of at least one audio source within an audio scene by analyzing respective two or more audio signals from the separate microphones, wherein the audio capture application is further configured to adaptively select, from the plurality of microphones, two or more respective audio signals based on the determined direction and furthermore configured to select, from the two or more respective audio signals, a reference audio signal also based on the determined direction; and a signal generator configured to generate a mid signal representing the at least one audio source based on a combination of the selected two or more respective audio signals and with reference to the reference audio signal.

First claim

The invention claimed is:

1. Apparatus comprising:
an audio capture application configured to determine a reference microphone signal from a plurality of microphones, wherein the reference microphone signal is provided from a reference microphone being closer to a sound source compared to at least one other microphone during an audio capturing, wherein the audio capture application is configured to select one or more microphones from the plurality of microphones based on the determined reference microphone so as to obtain one or more microphone signals, wherein the reference microphone and the one or more microphones are adaptively selected depending on the sound source position during the audio capturing, wherein the audio capture application is configured to determine delays between the selected one or more microphone signals and the reference microphone signal so as to time align each of the selected one or more microphone signals with the reference microphone signal, wherein the audio capture application is configured to process each microphone signal by a respective gain value, wherein the respective gain value is determined for each microphone position relative to the sound source during the audio capturing, wherein the audio capture application is configured to combine time aligned and processed microphone signals; and
a signal generator configured to generate a mid signal based on the combined time aligned and processed microphone signals.

2. The apparatus as claimed in claim 1 , wherein the audio capture application is further configured to:
identify two or more microphones from the plurality of microphones based on the determined direction and a microphone orientation such that the two or microphones identified are the microphones closest to the at least one audio source;
select based on the identified two or more microphones the two or more respective audio signals; and
identify from the two or microphones identified which microphone is closest to the at least one audio source based on the determined direction and configured to select the microphone closest to the at least one audio source respective audio signal as the reference audio signal.

3. The apparatus as claimed in claim 2 , wherein the audio capture application is further configured to determine a coherence delay between the reference audio signal and others of the selected two or more respective audio signals, wherein the coherence delay is the delay value which maximises the coherence between the reference audio signal and another of the two or more respective audio signals.

4. The apparatus as claimed in claim 1 , wherein the signal generator is configured to:
time align the others of the selected two or more respective audio signals with the reference audio signal based on the determined coherence delay;
combine the time aligned others of the selected two or more respective audio signals with the reference audio signal; and
generate a weighting value based on the difference between a microphone direction for the two or more respective audio signals and the determined direction, and further configured to apply the weighting value to the respective two or more audio signals prior to the signal generator combining.

5. The apparatus as claimed in claim 1 , further comprising a further signal generator configured to further select from the plurality of microphones, a further selection of two or more respective audio signals and generate from a combination of the further selection of two or more respective audio signals at least two side signals representing an audio scene ambience.

6. The apparatus as claimed in claim 5 , wherein the further signal generator is configured to select the further selection of two or more respective audio signals based on at least one of:
an output type; and
a distribution of the plurality of microphones.

7. The apparatus as claimed in claim 5 , wherein the further signal generator is configured to:
determine an ambience coefficient associated with each of the further selection of two or more respective audio signals;
apply the determined ambience coefficient to the further selection of two or more respective audio signals to generate a signal component for each of the at least two side signals; and
decorrelate the signal component for each of the at least two side signals.

8. The apparatus as claimed in claim 5 , wherein the further signal generator is configured to:
apply a pair of head related transfer function filters; and
combine the filtered decorrelated signal components to generate the at least two side signals representing the audio scene ambience; and
generate filtered decorrelated signal components to generate a left and a right channel audio signal representing the audio scene ambiance.

9. The apparatus as claimed in claim 5 , wherein the ambience coefficient for an audio signal from the further selection of two or more respective audio signals is based on a coherence value between the audio signal and the reference audio signal.

10. The apparatus as claimed in claim 5 , wherein the ambience coefficient for an audio signal from the further selection of two or more respective audio signals is based on at least one of:
a determined circular variance over time and/or frequency of a direction of arrival from the at least one audio source; and
both a coherence value between the audio signal and the reference audio signal and a determined circular variance over time and/or frequency of a direction of arrival from the at least one audio source.

11. A method comprising:
determining a reference microphone signal from a plurality of microphones, wherein the reference microphone signal is provided from a reference microphone being closer to a sound source compared to at least one other microphone during an audio capturing;
selecting one or more microphones from the plurality of microphones based on the determined reference microphone so as to obtain one or more microphone signals, wherein the reference microphone and the one or more microphones are adaptively selected depending on the sound source position during the audio capturing;
determining delays between the selected one or more microphone signals and the reference microphone signal so as to time align each of the selected one or more microphone signals with the reference microphone signal;
processing each microphone signal by a respective gain value, wherein the respective gain value is determined for each microphone position relative to the sound source during the audio capturing;
and
combining time aligned and processed microphone signals to generate a mid signal.

12. The method as claimed in claim 11 , wherein adaptively selecting, comprises:
identifying two or more microphones from the plurality of microphones based on the determined direction and a microphone orientation such that the two or microphones identified are the microphones closest to the at least one audio source; and
selecting based on the identified two or more microphones the two or more respective audio signals.

13. The method as claimed in claim 12 , wherein adaptively selecting, further comprises:
identifying from the two or microphones identified which microphone is closest to the at least one audio source based on the determined direction; and
selecting, from the two or more respective audio signals, a reference audio signal to select an audio signal associated with the microphone closest to the at least one audio source as the reference audio signal.

14. The method as claimed in claim 13 , further comprising determining a coherence delay between the reference audio signal

Assignees

• Nokia Technologies Oy

Inventors

• Laitinen Mikko-Ville
• Tammi Mikko
• Vilermo Miikka

Classifications

• H04R2201/401

  2D or 3D arrays of transducers · CPC title

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title

• H04S7/30

  Control circuits for electronic adaptation of the sound field · CPC title

• H04R3/005

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title