Optimal mixing matrices and usage of decorrelators in spatial audio processing

Patent metadata

Field	Value
Publication number	US-11282485-B2
Application number	US-202016987264-A
Country	US
Kind code	B2
Filing date	Aug 6, 2020
Priority date	Aug 17, 2011
Publication date	Mar 22, 2022
Grant date	Mar 22, 2022

Abstract

An apparatus for generating an audio output signal having two or more audio output channels from an audio input signal having two or more audio input channels includes a provider and a signal processor. The provider is adapted to provide first covariance properties of the audio input signal. The signal processor is adapted to generate the audio output signal by applying a mixing rule on at least two of the two or more audio input channels. The signal processor is configured to determine the mixing rule based on the first covariance properties of the audio input signal and based on second covariance properties of the audio output signal, the second covariance properties being different from the first covariance properties.

First claim

The invention claimed is:

1. An apparatus for generating an audio output signal comprising two or more audio output channels from an audio input signal comprising two or more audio input channels, comprising:
a provider for providing first covariance properties of the audio input signal, and
a signal processor for generating the audio output signal by applying a mixing rule on at least two of the two or more audio input channels,
wherein the signal processor is configured to determine the mixing rule based on the first covariance properties of the audio input signal and based on second covariance properties of the audio output signal, the second covariance properties being different from the first covariance properties.

2. The apparatus according to claim 1 , wherein the provider is adapted to provide the first covariance properties, wherein the first covariance properties comprise a first state for a first time-frequency bin, and wherein the first covariance properties comprise a second state, being different from the first state, for a second time-frequency bin, being different from the first time-frequency bin.

3. The apparatus according to claim 1 , wherein the signal processor is adapted to determine the mixing rule based on the second covariance properties, wherein the second covariance properties comprise a third state for a third time-frequency bin, and wherein the second covariance properties comprise a fourth state, being different from the third state for a fourth time-frequency bin, being different from the third time-frequency bin.

4. The apparatus according to claim 1 , wherein the signal processor is adapted to generate the audio output signal by applying the mixing rule such that each one of the two or more audio output channels depends on each one of the two or more audio input channels.

5. The apparatus according to claim 1 , wherein the signal processor is adapted to determine the mixing rule such that an error measure is minimized.

6. The apparatus according to claim 5 , wherein the signal processor is adapted to determine the mixing rule such that the mixing rule depends on
∥ y ref −y∥ 2 ,

wherein
y ref =Qx,
wherein x is the audio input signal, wherein Q is a mapping matrix, and wherein y is the audio output signal.

7. The apparatus according to claim 1 , wherein the signal processor is configured to determine the mixing rule by determining the second covariance properties, wherein the signal processor is configured to determine the second covariance properties based on the first covariance properties.

8. The apparatus according to claim 1 , wherein the signal processor is adapted to determine a mixing matrix as the mixing rule, wherein the signal processor is adapted to determine the mixing matrix based on the first covariance properties and based on the second covariance properties.

9. The apparatus according to claim 1 , wherein the provider is adapted to provide the first covariance properties by determining a first covariance matrix of the audio input signal, and wherein the signal processor is configured to determine the mixing rule based on a second covariance matrix of the audio output signal as the second covariance properties.

10. The apparatus according to claim 9 , wherein the provider is adapted to determine the first covariance matrix, such that each diagonal value of the first covariance matrix indicates an energy of one of the audio input channels, and such that each value of the first covariance matrix, which is not a diagonal value indicates an inter-channel correlation between a first audio input channel and a different second audio input channel.

11. The apparatus according to claim 9 , wherein the signal processor is configured to determine the mixing rule based on the second covariance matrix, wherein each diagonal value of the second covariance matrix indicates an energy of one of the audio output channels, and wherein each value of the second covariance matrix, which is not a diagonal value, indicates an inter-channel correlation between a first audio output channel and a second audio output channel.

12. The apparatus according to claim 1 , wherein the signal processor is adapted to determine a mixing matrix as the mixing rule, wherein the signal processor is adapted to determine the mixing matrix based on the first covariance properties and based on the second covariance properties, wherein the provider is adapted provide the first covariance properties by determining a first covariance matrix of the audio input signal, and wherein the signal processor is configured to determine the mixing rule based on a second covariance matrix of the audio output signal as the second covariance properties, wherein the signal processor is adapted to determine the mixing matrix such that:
M=K y PK x −1 ,
such that,
K x K x T =C x ,
K y K y T =C y
wherein M is the mixing matrix, wherein C x is the first covariance matrix, wherein C y is the second covariance matrix, wherein K x T is a first transposed matrix of a first decomposed matrix K x , wherein K y T is a second transposed matrix of a second decomposed matrix K y , wherein K x −1 is an inverse matrix of the first decomposed matrix K x , and wherein P is a first unitary matrix.

13. The apparatus according to claim 12 , wherein the signal processor is adapted to determine the mixing matrix such that
M=K y PK x −1 ,
wherein
P=VΛU T ,
wherein U T is a third transposed matrix of a second unitary matrix U, wherein V is a third unitary matrix, wherein Λ is an identity matrix appended with zeros, wherein
USV T =K x T Q x T K y ,

wherein Q T is a fourth transposed matrix of the mapping matrix Q,
wherein V T is a fifth transposed matrix of the third unitary matrix V, and wherein S is a diagonal matrix.

14. The apparatus according to claim 1 , wherein the signal processor is adapted to determine a mixing matrix as the mixing rule, wherein the signal processor is adapted to determine the mixing matrix based on the first covariance properties and based on the second covariance properties,
wherein the provider is adapted to provide the first covariance properties by determining a first covariance matrix of the audio input signal, and
wherein the signal processor is configured to determine the mixing rule based on a second covariance matrix of the audio output signal as the second covariance properties,
wherein the signal processor is adapted to determine the mixing rule by modifying at least some diagonal values of a diagonal matrix S x when the values of the diagonal matrix S x are zero or smaller than a threshold value, such that the values are greater than or equal to the threshold value,
wherein the diagonal matrix depends on the first covariance matrix.

15. The apparatus according to claim 14 , wherein the signal processor is configured to modify the at least some diagonal values of the diagonal matrix S x , wherein K x =U x S x V x T , and wherein C x =K x K x T , wherein C x is the first covariance matrix, wherein S x is the diagonal matrix, wherein U, is a second matrix, V x T is a third transposed matrix, and wherein K x T is a fourth transposed matrix of the fifth matrix K x , and wherein V x and U x are unitary matrices.

16. The apparatus according to claim 14 , wherein the signal processor is adapted to generate the audio output signal by applying the mixing matrix on at least two of the two or more audio input channels to acquire an intermediate signa

Assignees

• Fraunhofer Ges Forschung

Inventors

• Vilkamo Juha
• Baeckstroem Tom
• Kuech Fabian
• Kuntz Achim

Classifications

• G10L19/008 Primary

  Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing · CPC title

• G10L19/00

  Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis (in musical instruments G10H) · CPC title

• G10H1/183 Primary

  Channel-assigning means for polyphonic instruments · CPC title

• G10L19/0017 Primary

  Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error (G10L19/24 takes precedence) · CPC title