Sound enhancement through deverberation

Patent metadata

Field	Value
Publication number	US-9607627-B2
Application number	US-201514614793-A
Country	US
Kind code	B2
Filing date	Feb 5, 2015
Priority date	Feb 5, 2015
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

Sound enhancement techniques through dereverberation are described. In one or more implementations, a method is described of enhancing sound data through removal of reverberation from the sound data by one or more computing devices. The method includes obtaining a model that describes primary sound data that is to be utilized as a prior that assumes no prior knowledge about specifics of the sound data from which the reverberation is to be removed. A reverberation kernel is computed having parameters that, when applied to the model that describes the primary sound data, corresponds to the sound data from which the reverberation is to be removed. The reverberation is removed from the sound data using the reverberation kernel.

First claim

What is claimed is:

1. A method of enhancing sound data through removal of reverberation from the sound data by at least one computing devices, the method comprising:
obtaining, by the at least one computing device, a model that describes primary sound data that is to be utilized as a prior that assumes no prior knowledge about specifics of the sound data, captured by a sound capture device, from which the reverberation is to be removed;
computing, by the at least one computing device, a reverberation kernel based on the primary sound data and the sound data, the reverberation kernel having parameters that, when applied to the model that describes the primary sound data, corresponds to the sound data from which the reverberation is to be removed;
removing, by the at least one computing device, the reverberation from the sound data using the computed reverberation kernel; and
outputting, by the at least one computing device, the sound data having the removed reverberation.

2. A method as described in claim 1 , wherein the specifics are particular speakers or characteristics of a particular environment, in which, the sound data is captured.

3. A method as described in claim 1 , wherein the primary sound data is speech data that is generally clean and therefore generally free of noise.

4. A method as described in claim 1 , wherein the model is expressed as a set of latent variables of a probabilistic model.

5. A method as described in claim 4 , wherein the set of latent variables define a non-negative matrix factorization (NMF) model.

6. A method as described in claim 1 , wherein the computing of the reverberation kernel is performed using an expectation maximization (EM) algorithm to perform posterior inference.

7. A method as described in claim 1 , wherein the model is expressed as a product-of-filters model.

8. A method as described in claim 1 , further comprising:
estimating additive noise in the sound data as part of the computing of the reverberation kernel; and
removing additive noise based on the estimated additive noise from the sound data as part of the removing of the reverberation.

9. A method as described in claim 8 , wherein the computing of the reverberation kernel and the estimating of the additive noise are performed under a maximum-likelihood framework.

10. A method as described in claim 1 , wherein the computing includes attenuating a tail of the reverberation kernel.

11. A method of enhancing sound data through removal of noise from the sound data by at least one computing devices, the method comprising:
generating, by the at least one computing device, a model using non-negative matrix factorization (NMF) that describes primary sound data;
estimating, by the at least one computing device, additive noise and a reverberation kernel having parameters that, when applied to the model that describes the primary sound data, corresponds to the sound data from which reverberation is to be removed, the estimating based on the primary sound data and the sound data and the sound data captured by a sound capture device;
removing, by the at least one computing device, additive noise from the sound data based on the estimated additive noise and removing the reverberation from the sound data using the estimated reverberation kernel; and
outputting, by the at least one computing device, the sound data having the additive noise and the reverberation removed.

12. A method as described in claim 11 , wherein the model is to be utilized as a prior that assumes no prior knowledge about specifics of the sound data from which the reverberation is to be removed.

13. A method as described in claim 12 , wherein the specifics are particular speakers or characteristics of a particular environment, in which, the sound data is captured.

14. A method as described in claim 11 , wherein the estimating of the reverberation kernel is performed using an expectation maximization (EM) algorithm to perform posterior inference.

15. A method as described in claim 11 , wherein the estimating of the reverberation kernel and the estimating of the additive noise are performed under a maximum-likelihood framework.

16. A system of enhancing sound data through removal of reverberation from the sound data, the system comprising:
a model generation module implemented at least partially in hardware to generate a model that describes primary sound data that is to be utilized as a prior that assumes no prior knowledge about specifics of the sound data from which the reverberation is to be removed that is captured by a sound capture device;
a reverberation estimation module implemented at least partially in hardware to estimate a reverberation kernel having parameters based on the primary sound data and the sound data that, when applied to the model that describes the primary sound data, corresponds to the sound data from which the reverberation is to be removed; and
a noise removal module implemented at least partially in hardware to remove the reverberation from the sound data using the estimated reverberation kernel.

17. A system as described in claim 16 , wherein the specifics are particular speakers or characteristics of a particular environment, in which, the sound data is captured.

18. A system as described in claim 16 , wherein the model is expressed as a set of latent variables of a non-negative matrix factorization (NMF) model or a product-of-filters model.

19. A system as described in claim 16 , wherein the computing of the reverberation kernel is performed using an expectation maximization (EM) algorithm to perform posterior inference.

20. A system as described in claim 16 , further comprising an additive noise estimation module to estimate additive noise in the sound data as part of the computing of the reverberation kernel and remove additive noise from the sound data based on the estimated additive noise as part of the removal of the reverberation.

Assignees

• Adobe Systems Inc

Inventors

• Liang Dawen
• Hoffman Matthew Douglas
• Mysore Gautham J

Classifications

• G10L21/0216

  characterised by the method used for estimating noise · CPC title

• G10L2021/02082

  the noise being echo, reverberation of the speech · CPC title

• G10L21/0232

  Processing in the frequency domain · CPC title

• G10L21/0208 Primary

  Noise filtering · CPC title