System and method for assisting selective hearing

Patent metadata

Field	Value
Publication number	US-12069470-B2
Application number	US-202217665379-A
Country	US
Kind code	B2
Filing date	Feb 4, 2022
Priority date	Aug 6, 2019
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

A system and a corresponding method for assisting selective hearing are provided. The system includes a detector for detecting an audio source signal portion of one or more audio sources by using at least two received microphone signals of a hearing environment. In addition, the system includes a position determiner for allocating position information to each of the one or more audio sources. In addition, the system includes an audio type classifier for assigning an audio source signal type to the audio source signal portion of each of the one or more audio sources. In addition, the system includes a signal portion modifier for varying the audio source signal portion of at least one audio source of the one or more audio sources depending on the audio signal type of the audio source signal portion of the at least one audio source so as to obtain a modified audio signal portion of the at least one audio source. In addition, the system includes a signal generator.

First claim

The invention claimed is:

1. System for assisting selective hearing, the system comprising:
a detector for detecting an audio source signal portion of one or more audio sources by using at least two received microphone signals of a hearing environment,
a position determiner for assigning position information to each of the one or more audio sources,
an audio type classifier for allocating an audio signal type to the audio source signal portion of each of the one or more audio sources,
a signal portion modifier for varying the audio source signal portion of at least one audio source of the one or more audio sources depending on the audio signal type of the audio source signal portion of the at least one audio source so as to acquire a modified audio signal portion of the at least one audio source, and
a signal generator for generating a plurality of binaural room impulse responses for each audio source of the one or more audio sources depending on the position information of this audio source and an orientation of a user's head, and for generating at least two loudspeaker signals depending on the plurality of the binaural room impulse responses and depending on the modified audio signal portion of the at least one audio source.

2. System according to claim 1 ,
wherein the detector is configured to detect the audio source signal portion of the one or more audio sources by using deep learning models.

3. System according to claim 1 ,
wherein the positon determiner is configured to determine, for each of the one or more audio sources, the position information depending on a captured image or a recorded video.

4. System according to claim 3 ,
wherein the position determiner is configured to determine, for each of the one or more audio sources, the position information depending on the video by detecting a lip movement of a person in the video and by allocating, depending on the lip movement, the same to the audio source signal portion of one of the one or more audio sources.

5. System according to claim 1 ,
wherein the detector is configured to determine one or more acoustical properties of the hearing environment depending on the at least two received microphone signals.

6. System according to claim 5 ,
wherein the signal generator is configured to determine the plurality of binaural room impulse responses depending on the one or more acoustical properties of the hearing environment.

7. System according to claim 1 ,
wherein the signal portion modifier is configured to select the at least one audio source whose audio source signal portion is modified, depending on a previously learned user scenario and to modify the same depending on the previously learned user scenario.

8. System according to claim 7 , wherein the system comprises a user interface for selecting the previously learned user scenario from a group of two or more previously learned user scenarios.

9. System according to claim 1 ,
wherein the detector and/or the position determiner and/or the audio type classifier and/or the signal portion modifier and/or the signal generator is configured to perform parallel signal processing using a Hough transformation or employing a plurality of VLSI chips or by employing a plurality of memristors.

10. System according to claim 1 ,
wherein a system comprises a hearing device that serves as a hearing aid for users that are limited in their hearing capability and/or have damaged hearing, wherein the hearing device comprises at least two loudspeakers for outputting the at least two loudspeaker signals.

11. System according to claim 1 ,
wherein the system comprises at least two loudspeakers for outputting the at least two loudspeaker signals, and a housing structure that houses the at least two loudspeakers, wherein the at least one housing structure is suitable for being fixed to a user's head or to any other body part of the user.

12. System according to claim 1 ,
wherein the system comprises a headphone that comprises at least two loudspeakers for outputting the at least two loudspeaker signals.

13. System according to claim 12 , wherein the detector and the position determiner and the audio classifier and the signal portion modifier and the signal generator are integrated into the headphone.

14. System according to claim 12 ,
wherein the system comprises a remote device that comprises the detector and the position determiner and the audio type classifier and the signal portion modifier and the signal generator,
wherein the remote device is spatially separated from the headphone.

15. System according to claim 14 , wherein the remote device is a smartphone.

16. System according to claim 1 ,
wherein the system further comprises an apparatus for determining one or more room acoustics parameters.

17. System according to claim 16 ,
wherein the signal portion modifier is configured to perform the variation of the audio source signal portion of the at least one audio source of the one or more audio sources depending on at least one of the one or more room acoustics parameters; and/or
wherein the signal generator is configured to perform the generation of at least one of the plurality of binaural room impulse responses for each audio source of the one or more audio source depending on the at least one of the one or more room acoustics parameters.

18. Method for assisting selective hearing, the method comprising:
detecting an audio source signal portion of one or more audio sources by using at least two received microphone signals of a hearing environment,
assigning position information to each of the one or more audio sources,
allocating an audio signal type to the audio source signal portion of each of the one or more audio sources,
varying the audio source signal portion of at least one audio source of the one or more audio sources depending on the audio signal type of the audio source signal portion of the at least one audio source so as to acquire a modified audio signal portion of the at least one audio source, and
generating a plurality of binaural room impulse responses for each audio source of the one or more audio sources depending on the position information of this audio source and an orientation of a user's head, and for generating at least two loudspeaker signals depending on the plurality of the binaural room impulse responses and depending on the modified audio signal portion of the at least one audio source.

19. A non-transitory digital storage medium having a computer program stored thereon to perform the method for assisting selective hearing, the method comprising:
detecting an audio source signal portion of one or more audio sources by using at least two received microphone signals of a hearing environment,
assigning position information to each of the one or more audio sources,
allocating an audio signal type to the audio source signal portion of each of the one or more audio sources,
varying the audio source signal portion of at least one audio source of the one or more audio sources depending on the audio signal type of the audio source signal portion of the at least one audio source so as to acquire a modified audio signal portion of the at least one audio source, and
generating a plurality of binaural room impulse responses for each audio source of the one or more audio sources depending on the position information of this audio source and an orientation of a user's head, and for generating at least two loudspeaker signals depending on the plurality of the binaural room impulse respon

Assignees

• Fraunhofer Ges Forschung
• Univ Ilmenau Tech

Inventors

• Sporer Thomas
• Fischer Georg
• Lukashevich Hanna
• Klein Florian
• Werner Stephan
• Neidhardt Annika
• Sloma Ulrike
• Schneiderwind Christian
• Stirnat Claudia
• Cano Ceron Estefania
• Abesser Jakob
• Sladeczek Christoph
• Brandenburg Karlheinz

Classifications

• H04R5/04

  Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} · CPC title

• H04R5/033

  Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)} · CPC title

• H04R3/005

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title