Multi-sourced noise suppression

Patent metadata

Field	Value
Publication number	US-9799330-B2
Application number	US-201514838133-A
Country	US
Kind code	B2
Filing date	Aug 27, 2015
Priority date	Aug 28, 2014
Publication date	Oct 24, 2017
Grant date	Oct 24, 2017

Abstract

Systems and methods for multi-sourced noise suppression are provided. An example system may receive streams of audio data including a voice signal and noise, the voice signal including a spoken word. The streams of audio data are provided by distributed audio devices. The system can assign weights to the audio streams based at least partially on quality of the audio streams. The weights of audio streams can be determined based on signal-to-noise ratios (SNRs). The system may further process, based on the weights, the audio stream to generate cleaned speech. Each audio device comprises microphone(s) and can be associated with the Internet of Things (IoT), such that the audio devices are Internet of Things devices. The processing can include noise suppression and reduction and echo cancellation. The cleaned speech can be provided to a remote device for further processing which may include Automatic Speech Recognition (ASR).

First claim

What is claimed is:

1. A method for multi-sourced noise suppression, the method comprising:
assigning weights to audio streams, the audio streams being provided substantially synchronously by a plurality of audio devices, the weights depending on quality of the audio streams, wherein the assigning weights includes generating an acoustic activity map by locating, identifying and mapping target sounds and noise sources in at least one of a single room and multi-room environment, so as to create a multidimensional acoustic view of the environment;
based on the weights, performing noise suppression processing on the audio streams to generate a cleaned voice signal;
providing the cleaned voice signal from the noise suppression processing to at least one remote device for further processing; and
based on the acoustic activity map, selecting an optimal one of the plurality of audio devices to communicate with the user.

2. The method of claim 1 , wherein each of the audio devices includes at least one microphone and the audio devices are connected in a dynamic network of connected devices, such that the audio devices are connected as part of an Internet of Things environment.

3. The method of claim 1 , wherein the weights are proportional to at least one quality metric for the audio stream, the quality metric comprising at least one signal to noise ratio (SNR).

4. The method of claim 1 , wherein the performing of the noise suppression processing is provided in combination with at least one of:
performing noise reduction; and
performing echo cancellation.

5. The method of claim 1 , further including continually updating acoustic signatures between the audio devices based on one or more sound sources located in the vicinity of the audio devices.

6. The method of claim 1 , wherein auditory scene analysis and scene classifiers are used for the identifying of target sounds and noise sources.

7. The method of claim 1 , wherein the audio streams include time stamps, the method further comprising, based on the time stamps, synchronizing the audio devices to a common time source.

8. The method of claim 7 , further comprising, based on the acoustic activity map, assigning weights to the audio streams based on the SNR quality metric.

9. The method of claim 8 , further comprising, based on the acoustic activity map, assigning weights to the audio streams further based on the degree to which the associated audio device, that provides a respective one of the audio streams, measures noise.

10. The method of claim 1 , wherein the communication with the user is via a loudspeaker of the optimal audio device.

11. The method of claim 7 , wherein the performing of the noise suppression processing is provided in combination with at least one of:
performing noise reduction; and
performing echo cancellation.

12. The method of claim 1 , wherein the audio streams comprise at least one voice command to perform at least one of activating the remote device and communicating with another user.

13. The method of claim 1 , wherein the further processing comprises automatic speech recognition (ASR) processing of the cleaned voice signal.

14. The method of claim 13 , wherein, based on the ASR processing, a context of a command to connect to another user is recognized and the cleaned voice signal is communicated to at least one of the audio devices located in proximity to the other user for establishing two way communication therewith.

15. A system for multi-sourced audio processing, the system comprising:
a processor; and
a memory communicatively coupled with the processor, the memory storing instructions, which, when executed by the processor, perform a method comprising:
assigning weights to audio streams, the audio streams being provided substantially synchronously by a plurality of audio devices, the weights depending on quality of the audio streams;
based on the weights, performing noise suppression processing on the audio streams to generate a cleaned voice signal, and
providing the cleaned voice signal from the noise suppression processing to a remote device for further processing,

wherein each of the audio devices includes at least one microphone and wherein the plurality of audio devices are physically separate from each other but connected in a dynamic network of connected devices, such that the audio devices are connected as part of an Internet of Things environment.

16. The system of claim 15 , wherein the assigning weights includes generating an acoustic activity map by locating, identifying and mapping target sounds and noise sources in at least one of a single room and multi-room environment, so as to create a multidimensional acoustic view of the environment.

17. A non-transitory computer-readable storage medium having embodied thereon instructions, which, when executed by at least one processor, perform steps of a method, the method comprising:
assigning weights to audio streams, the audio streams being provided substantially synchronously by a plurality of audio devices, the weights depending on quality of the audio streams, wherein the assigning weights includes generating an acoustic activity map by locating, identifying and mapping target sounds and noise sources in at least one of a single room and multi-room environment, so as to create a multidimensional acoustic view of the environment;
based on the weights, performing noise suppression processing on the audio streams to generate a cleaned voice signal;
providing the cleaned voice signal from the noise suppression processing to at least one remote device for further processing; and
based on the acoustic activity map, selecting an optimal one of the plurality of audio devices to communicate with the user.

18. The method of claim 1 , wherein one or more of the plurality of audio devices is incorporated in an Internet of Things device.

19. The non-transitory computer-readable medium of claim 17 , wherein one or more of the plurality of audio devices is incorporated in an Internet of Things device.

20. The system of claim 15 , further comprising:
a controller for receiving the audio streams from the plurality of audio devices via the dynamic network and for performing the noise suppression processing on the received audio streams.

Assignees

• Knowles Electronics Llc

Inventors

• Nemala Sridhar Krishna
• Woodruff John
• Verma Tony
• Caldwell Frederic

Classifications

• H04R3/005

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• G10L2021/02082

  the noise being echo, reverberation of the speech · CPC title

• G10L25/06

  the extracted parameters being correlation coefficients · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G10L21/0216

  characterised by the method used for estimating noise · CPC title