Exploiting visual information for enhancing audio signals via source separation and beamforming

Patent metadata

Field	Value
Publication number	US-11295137-B2
Application number	US-202017086561-A
Country	US
Kind code	B2
Filing date	Nov 2, 2020
Priority date	Jun 11, 2014
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

A system for exploiting visual information for enhancing audio signals via source separation and beamforming is disclosed. The system may obtain visual content associated with an environment of a user, and may extract, from the visual content, metadata associated with the environment. The system may determine a location of the user based on the extracted metadata. Additionally, the system may load, based on the location, an audio profile corresponding to the location of the user. The system may also load a user profile of the user that includes audio data associated with the user. Furthermore, the system may cancel, based on the audio profile and user profile, noise from the environment of the user. Moreover, the system may include adjusting, based on the audio profile and user profile, an audio signal generated by the user so as to enhance the audio signal during a communications session of the user.

First claim

What is claimed is:

1. A method, comprising:
determining, by a user equipment comprising a processor, metadata of media captured in an environment in which a communication session is being performed using the user equipment, wherein the metadata comprises information indicating classifying what a user in the environment is doing and an interferer generating noise in the environment; and
cancelling, by the user equipment, the noise in the environment based on the metadata and a user profile associated with the user.

2. The method of claim 1 , wherein the cancelling is further based on an audio profile associated with the interferer.

3. The method of claim 1 , wherein the metadata further comprises a location of the environment.

4. Then method of claim 3 , wherein the cancelling is further based on an audio profile associated with the location.

5. The method of claim 1 , wherein the metadata comprises a location of the interferer in the environment.

6. The method of claim 5 , wherein the cancelling further comprises transmitting a cancellation signal in a direction of the location of the interferer, and wherein the cancellation signal corresponds to the noise.

7. The method of claim 1 , further comprising adjusting, by the user equipment, an audio signal received from the user based on the user profile.

8. A user equipment, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
identifying metadata of media captured in an environment in which an audio conversation is being performed using the user equipment, wherein the metadata describes at least what a user determined to be in the environment is doing and an interferer generating noise in the environment; and
suppressing, by the user equipment, the noise in the environment based on the metadata and a user profile associated with the user.

9. The user equipment of claim 8 , wherein the suppressing is further based on an audio profile associated with the interferer.

10. The user equipment of claim 8 , wherein the metadata further comprises a location of the environment.

11. The user equipment of claim 10 , wherein the suppressing is further based on an audio profile associated with the location.

12. The user equipment of claim 8 , wherein the metadata comprises a location of the interferer in the environment.

13. The user equipment of claim 12 , wherein the suppressing further comprises transmitting a cancellation signal in a direction of the location of the interferer, and wherein the cancellation signal corresponds to the noise.

14. The user equipment of claim 8 , wherein the operations further comprise enhancing, by the user equipment, an audio signal that was generated by and received from the user based on the user profile.

15. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor of a user equipment, facilitate performance of operations, comprising:
extracting metadata from media captured in an environment in which a communication session is being performed via the user equipment, wherein the metadata indicates a user act taking place in the environment and an interferer generating noise in the environment; and
suppressing the noise in the environment based on the metadata and a user profile associated with a user identity determined to be associated with the user act.

16. The non-transitory machine-readable medium of claim 15 , wherein the suppressing is further based on an audio profile associated with the interferer.

17. The non-transitory machine-readable medium of claim 15 , wherein the metadata further comprises a location of the environment.

18. The non-transitory machine-readable medium of claim 17 , wherein the suppressing is further based on an audio profile associated with the location.

19. The non-transitory machine-readable medium of claim 15 , wherein the metadata comprises a location of the interferer in the environment.

20. The non-transitory machine-readable medium of claim 19 , wherein the suppressing further comprises transmitting a cancellation signal in a direction of the location of the interferer, and wherein the cancellation signal corresponds to the noise.

Assignees

• At & T Ip I Lp
• At&T Iniellectual Property I L P

Inventors

• Dimitriadis Dimitrios
• Bowen Donald J
• Ji Lusheng
• Schroeter Horst J

Classifications

• H04R2430/20

  Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic (H04R2203/12 takes precedence) · CPC title

• H04R2460/07

  Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection · CPC title

• H04R5/04 Primary

  Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} · CPC title

• G10L21/0208

  Noise filtering · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title