PSD optimization apparatus, PSD optimization method, and program

Patent metadata

Field	Value
Publication number	US-11922964-B2
Application number	US-201917633191-A
Country	US
Kind code	B2
Filing date	Aug 8, 2019
Priority date	Aug 8, 2019
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

Sound source enhancement technology is provided that is capable of improving sound source enhancement capabilities in accordance with settings of usage and applications. A PSD optimization device includes a PSD updating unit that takes a target sound PSD input value {circumflex over ( )}φS(ω, τ), an interference noise PSD input value {circumflex over ( )}φIN(ω, τ), and a background noise PSD input value {circumflex over ( )}φBN(ω, τ) as input, and generates a target sound PSD output value φS(ω, τ), an interference noise PSD output value {circumflex over ( )}φIN(ω, τ), and a background noise PSD output value {circumflex over ( )}φBN(ω, τ), by solving an optimization problem for a cost function relating to a variable uS representing a target sound PSD, a variable uIN representing an interference noise PSD, and a variable uBN representing a background noise PSD. The optimization problem for the cost function is defined using at least one of a constraint relating to a frequency structure of a sound source or a convex cost term relating to the frequency structure of the sound source, a constraint relating to a temporal structure of the sound source or a convex cost term relating to the temporal structure of the sound source, and a constraint relating to a spatial structure of the sound source or a convex cost term relating to the spatial structure of the sound source.

First claim

The invention claimed is:

1. A power spectral density (PSD) optimization device including a PSD updating unit that, with us as a variable representing a target sound PSD, u IN as a variable representing an interference noise PSD, and u BN as a variable representing a background noise PSD, takes a target sound PSD input value {circumflex over ( )}φ S (ω, τ), an interference noise PSD input value {circumflex over ( )}φ IN (ω, τ), and a background noise PSD input value {circumflex over ( )}φ BN (ω, τ) as input, and generates a target sound PSD output value φ S (ω, τ), an interference noise PSD output value φ IN (ω, τ), and a background noise PSD output value φ BN (ω, τ), by solving an optimization problem for a cost function relating to the variable u S , the variable u IN , and the variable u BN ,
wherein the optimization problem for the cost function is defined using at least one of
a constraint relating to a frequency structure of a sound source or a convex cost term relating to the frequency structure of the sound source,
a constraint relating to a temporal structure of the sound source or a convex cost term relating to the temporal structure of the sound source, and
a constraint relating to a spatial structure of the sound source or a convex cost term relating to the spatial structure of the sound source.

2. The PSD optimization device according to claim 1 ,
wherein a convex cost term L relating to the frequency structure is defined by

L
⁡
(

u
S

)

=

μ
⁢



Λ
⁢

u
S



1

+

ρ
2

⁢



Λ
⁢

u
S

-

Λ
⁢

φ
^

S



2
2

where μ, ρ(∈R + ) are weighting coefficients, Λ(∈R Ω×Ω ) is a predetermined sparse matrix, and Ω is a count of frequency bands.

3. The PSD optimization device according to claim 1 ,
wherein, with u BN,τ as a variable representing the background noise PSD at a time frame τ, and {circumflex over ( )}φ BN,τ−1 as a background noise PSD estimation value at time frame τ−1,
a convex cost term L relating to the temporal structure is defined by

L
⁡
(

u

BN
,
τ

)

=

γ
BN

2

⁢



φ
^

BN
,

τ
-
1

-

u

BN
,
τ



2
2

where γ BN (∈R + ) is a weighting coefficient.

4. The PSD optimization device according to claim 1 ,
wherein, with c as a PSD estimation value of enhanced signals of the sound source at the target sound direction-of-arrival θ S ,
the constraint relating to the spatial structure is defined by
u S +u IN +u BN =c.

5. The PSD optimization device according to claim 1 ,
wherein, with u=[u S T , u IN T , u BN T ] T , and v as an auxiliary variable of variable u,
the optimization problem for the cost function is defined as a problem in which inf u,v F 1 (u)+F 2 (v) (where F 1 and F 2 are convex functions) is solved under linear constraints relating to the variables u and v,
and where the linear constraints relating to the variables u and v include at least one of a constraint relating to the frequency structure of the sound source, a constraint relating to the temporal structure of the sound source, and a constraint relating to the spatial structure of the sound source, or F 1 (u)+F 2 (v) includes at least one of a convex cost term relating to the frequency structure of the sound source, a convex cost term relating to the temporal structure of the sound source, and a convex cost term relating to the spatial structure of the sound source.

6. The PSD optimization device according to claim 5 ,
wherein linear constraints regarding the variables u and v are
Au=v,Bu=c,u≥ 0

where A=[Λ 0 0], B=[I, I, I], c is the PSD estimation value of enhanced signals of the sound source at the target sound direction-of-arrival θ S , Λ(∈R Ω×Ω ) is a predetermined sparse matrix, I(∈R Ω×Ω ) is an identity matrix, and Ω is the number of frequency bands,
F 1 (u) and F 2 (v) are each

F
1

(
u
)

=

1
2

⁢



W

1
/
2

(

Assignees

• Nippon Telegraph & Telephone

Inventors

• Chiba Hironobu
• Niwa Kenta
• Harada Noboru

Classifications

• G10L21/0232 Primary

  Processing in the frequency domain · CPC title

• G06F17/11

  for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title

• G10L2021/02166

  Microphone arrays; Beamforming · CPC title

• H04R3/04

  for correcting frequency response · CPC title

• H04R3/00

  Circuits for transducers (arrangements for producing a reverberation or echo sound G10K15/08; amplifiers H03F) · CPC title