Hearing device comprising a low-latency sound source separation unit

Patent metadata

Field	Value
Publication number	US-10341785-B2
Application number	US-201514874641-A
Country	US
Kind code	B2
Filing date	Oct 5, 2015
Priority date	Oct 6, 2014
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

The application relates to a hearing device comprising a) an input unit for delivering a time varying electric input signal representing an audio signal comprising at least two sound sources, b) a cyclic analysis buffer unit of length A adapted for storing the last A audio samples, c) a cyclic synthesis buffer unit of length, where L is smaller than A, adapted for storing the last L audio samples, which are intended to be separated in individual sound sources, d) a database having stored recorded sound examples from said at least two sound sources, each entry in the database being termed an atom, the atoms originating from audio samples from first and second buffers corresponding in size to said synthesis and analysis buffer units, where for each atom, the audio samples from the first buffer overlaps with the audio samples from the second buffer, and where atoms originating from the first buffer constitute a reconstruction dictionary, and where atoms originating from the second buffer constitute an analysis dictionary. The application further relates to a method of separating audio sources, and e) a sound source separation unit for separating said electric input signal to provide separated signals representing said at least two sound sources, the sound source separation unit being configured to determine the most optimal representation (W) of the last A samples given the atoms in the analysis dictionary of the database, and to generate said at least two sound sources by combining atoms in the reconstruction dictionary of the database using the optimal representation (W). The invention may e.g. be used for hearing devices, e.g. hearing aids, headsets, ear phones, active ear protection systems, handsfree telephone systems, mobile telephones, teleconferencing systems, public address systems, classroom amplification systems, etc.

First claim

The invention claimed is:

1. A hearing device comprising:
an input unit for delivering a time varying electric input signal representing an observed audio signal comprising at least two sound sources,
a cyclic analysis buffer unit of length A adapted for storing the last A audio samples,
a cyclic synthesis buffer unit of length L, where L is smaller than A, adapted for storing the last L audio samples, which are intended to be separated in individual sound sources,
a database storing an analysis dictionary and a reconstruction dictionary of recorded sound examples from each of the at least two sound sources, each recorded sound example in the database being termed an atom, wherein the reconstruction dictionary includes atoms, from each of the at least two sound sources, originating from audio samples from a first buffer of length L and the analysis dictionary includes atoms, from each of the at least two sound sources, originating from audio samples from a second buffer of length A, where for each atom, the audio samples from the first buffer overlap with the audio samples from the second buffer such that audio samples from first and second buffers form atom pairs between the analysis and reconstruction dictionaries,
a sound source separation unit for separating said electric input signal to provide at least two separated signals representing said at least two sound sources, the sound source separation unit being configured to
estimate the observed audio signal as a weighted summation of the atoms in the dictionaries stored in the database,
determine an optimal weight representation (W) of the last A audio samples of the observed audio signal by minimizing a cost function between the samples of the observed audio signal and the estimated signal given the atoms in the analysis dictionary of the database, and
generate said at least two separated signals of L audio samples by combining atoms in the reconstruction dictionary of the database using the optimal weight representation (W).

2. A hearing device according to claim 1 comprising
a time frequency conversion unit for providing the contents of said analysis buffer units in a time-frequency representation (k,m), wherein the corresponding time segment of the electric input signal is provided in a number of frequency bands at a number of time instances, k being a frequency band index and m being a time index, and wherein (k,m) defines a specific time-frequency bin or unit comprising a signal component in the form of a complex or real value of the electric input signal corresponding to frequency index k and time instance m.

3. A hearing device according to claim 2 comprising a time-frequency to time conversion unit for providing the time domain representation of the separated sources.

4. A hearing device according to claim 1 comprising a feature extraction unit for extracting characteristic features of the contents of said analysis buffer unit and said synthesis buffer unit.

5. A hearing device according to claim 1 wherein the sound separation unit is configured to base said sound source separation on Non Negative Matrix Factorization (NMF), Hidden Markov Model (HMM), or Deep Neural Networks (DNN).

6. A hearing device according to claim 1 wherein each corresponding atom pair originating from audio samples from first and second buffers of said database comprises an identifier of the sound source from which it originates.

7. A hearing device according to claim 6 wherein the sound source separation unit is configured to use the identifier of the sound source to generate said at least two sound sources.

8. A hearing device according to claim 1 comprising a control unit for controlling the update of the analysis and synthesis buffer units with a predefined update frequency, and configured—at each update—to store in the analysis and synthesis buffers the last H audio samples received from the input unit and discarding the oldest H audio samples stored in the analysis and synthesis buffer units.

9. A hearing device according to claim 1 comprising a signal processing unit for processing one or more of said separated signals representing said at least two sound sources.

10. A hearing device according to claim 1 comprising a directional microphone system.

11. A hearing device according to claim 1 which for each of said at least two sound sources comprise a separate dictionary for the purposes of analysis and reconstruction, respectively.

12. A hearing device according to claim 1 comprising a hearing aid, a headset, an ear phone, an active ear protection systems or a combination thereof.

13. A hearing device according to claim 1 , wherein the functional components of the hearing device are enclosed in a single device.

14. A hearing device according to claim 1 , wherein the functional components of the hearing device are enclosed in several separate devices.

15. A hearing device according to claim 14 , wherein the several separate devices are adapted to be in wired or wireless communication with each other.

16. A hearing device according to claim 1 comprising a hearing instrument and an auxiliary device configured to allow an exchange of data between them.

17. A hearing device according to claim 16 wherein the hearing instrument is an ear worn hearing instrument and processing demanding tasks are performed in the auxiliary device having more electric power and processing capacity than the ear worn hearing instrument.

18. A hearing device according to claim 16 wherein at least a part of the database is located in the auxiliary device.

19. A hearing device according to claim 16 wherein the sound source separation unit is located in the auxiliary device.

20. A hearing device according to claim 16 wherein the auxiliary device is or comprises a remote control or other hand held device.

21. A hearing device according to claim 20 wherein the remote control is implemented in a smartphone, the smartphone running an application allowing the user to control the functionality of the hearing device via the smartphone, the hearing device comprising an appropriate wireless interface to the smartphone.

22. A hearing device according to claim 16 wherein the auxiliary device comprises the input unit for receiving an input sound signal comprising a multitude of sound sources and providing a digitized electric input signal representing a mixed sound signal.

23. A hearing device according to claim 16 wherein the auxiliary device comprises a signal processing unit for processing one or more of the separated signals.

24. A hearing device according to claim 23 wherein the processed output is transferred to the hearing instrument over a wireless connection implemented by corresponding antenna and transceiver circuitry in the auxiliary device and the hearing instrument.

25. A hearing device according to claim 24 wherein the hearing instrument is configured to receive the processed output signal and to present the signal to a user via an output unit as a sound signal.

26. A hearing device according to claim 16 wherein the auxiliary device comprises a user interface.

27. A hearing device according to claim 26 wherein the user interface is configured to display currently present sound sources.

28. A hearing device according to claim 27 wherein the user interface is configured to display a position relative to a user of the curre

Assignees

• Oticon As

Inventors

• Barker Thomas
• Virtanen Tuomas
• Pontoppidan Niels Henrik

Classifications

• G10L21/028

  using properties of sound source · CPC title

• H04R25/505 Primary

  using digital signal processing · CPC title

• H04S1/005

  For headphones · CPC title

• H04R2225/43

  Signal processing in hearing aids to enhance the speech intelligibility · CPC title

• G10L21/0272 Primary

  Voice signal separating · CPC title