Real-time augmented hearing platform

Patent metadata

Field	Value
Publication number	US-11217268-B2
Application number	US-201916675976-A
Country	US
Kind code	B2
Filing date	Nov 6, 2019
Priority date	Nov 6, 2019
Publication date	Jan 4, 2022
Grant date	Jan 4, 2022

Abstract

An audio system and method for generating improved augmented hearing effects in real-time. The system includes a wearable audio device arranged to obtain audio that includes a voice signal associated with a user and an environmental signal associated with other audio picked up by the wearable audio device that does not includes the user's voice. In some implementations, at least one processor of the wearable audio device is configured to isolate the voice signal from the environmental signal such that augmented hearing effects can be applied to the voice signal and/or the environmental signal, independently. In some implementations, augmented hearing effects are designated by a signal augmentation profile generated or stored in the wearable audio device such that the total time between receipt of the audio signal and the modifications and/or transformations to the respective audio signals does not exceed 100 ms.

First claim

What is claimed is:

1. A wearable audio device for modifying an audio signal, comprising:
at least one microphone arranged to receive an audio signal comprising a voice signal of a user wearing the wearable audio device and an environmental signal;
at least one audio output device arranged on or in to the wearable audio device, the at least one audio output device arranged to generate an audio output signal; and
at least one processor configured to:
receive the audio signal from the at least one microphone;
isolate the voice signal from the environmental signal;
modify the voice signal and/or the environmental signal; and
generate the audio output signal, wherein the audio output signal comprises a mixed audio signal comprising
i) the modified voice signal with the environmental signal;
ii) the voice signal with the modified environmental signal; or
iii) the modified voice signal with the modified environmental signal.

2. The wearable audio device of claim 1 , wherein a total time period between the receipt of the audio signal from the at least one microphone to the generation of the audio output signal is less than or equal to 100 milliseconds.

3. The wearable audio device of claim 1 , wherein the wearable audio device is arranged to receive a signal augmentation profile, wherein the signal augmentation profile is used by the processor when isolating the voice signal from the environmental signal and modifying the voice signal and/or the environmental signal.

4. The wearable audio device of claim 3 , wherein the signal augmentation profile is received at a first time, and the step of isolating the voice signal from the environmental signal is conducted at a second time after the first time.

5. The wearable audio device of claim 1 , wherein the modified voice signal is provided by a first audio channel.

6. The wearable audio device of claim 5 , wherein the modified environmental signal is provided by a second audio channel different from the first audio channel.

7. The wearable audio device of claim 1 , wherein the modification of the voice signal and/or the environmental signal includes modification of at least one of: a frequency-shift, a time-shift, a spatialization-shift, a gain modification, an equalization modification, an echo modification, an auto-tune modification, or a reverberation modification.

8. The wearable audio device of claim 1 , wherein the processor is further configured to, in response to user input, switch the wearable audio device to a non-modified state to a modified state, wherein the non-modified state includes generating the audio output signal without modification or with active noise reduction or active noise cancellation.

9. The wearable audio device of claim 1 , wherein the processor is further configured to:
identify an audio event within the audio signal, wherein the audio event corresponds to a predefined or preset noise signature;
transform audio associated with the audio event; and
generate the audio output signal, wherein the audio output signal further comprises the transformed audio associated with the audio event.

10. The wearable audio device of claim 9 , wherein the audio associated with the audio event is not generated in the audio output signal, such that it is completely replaced by the transformed audio associated with that audio event.

11. The wearable audio device of claim 1 , wherein the wearable audio device further comprises an active noise reduction (ANR) module or a noise cancelling (NC) module.

12. A method for modifying an audio signal, comprising:
receiving, via at least one microphone of a wearable audio device, an audio signal comprising a voice signal of a user wearing the wearable audio device and an environmental signal;
isolating, via a processor, the voice signal from the environmental signal;
modifying, using the processor, the voice signal and/or the environmental signal; and
generating, via at least one audio output device arranged on, in, or in proximity to the wearable audio device, an audio output signal, wherein the audio output signal comprises a mixed audio signal comprising i) the modified voice signal with the environmental signal; ii) the voice signal with the modified environmental signal; or iii) the modified voice signal with the modified environmental signal.

13. The method of claim 12 , wherein a total time period between the receipt of the audio signal from the at least one microphone to the generation of the audio output signal is less than or equal to 100 milliseconds.

14. The method of claim 12 , wherein the wearable audio device is arranged to receive a signal augmentation profile at a first time, wherein the signal augmentation profile used by the processor when isolating the voice signal at a second time after the first time from the environmental signal and modifying the voice signal and/or the environmental signal.

15. The method of claim 12 , wherein the modified voice signal is provided by a first audio channel and the modified environmental signal is provided by a second audio channel different than the first audio channel.

16. The method of claim 12 , wherein the modification of the voice signal and/or the environmental signal includes modification of at least one of: a frequency-shift, a time-shift, a spatialization-shift, a gain modification, an equalization modification, an echo modification, an auto-tune modification, or a reverberation modification.

17. The method of claim 12 , wherein the audio signal may include an audio event signal associated with an audio event and wherein processor is further arranged to:
identify the audio event from the audio event signal, wherein the audio event corresponds to a predefined or preset noise signature;
transform the audio event signal associated with the audio event; and
generate the audio output signal, wherein the audio output signal further comprises the transformed audio associated with the audio event signal.

18. A computer program product stored on a non-transitory computer-readable medium which includes a set of non-transitory computer-readable instructions for modifying an audio signal, that when executed on a processor of a wearable audio device is arranged to:
receive, via at least one microphone, an audio signal from an environment comprising a voice signal of a user wearing the wearable audio device and an environmental signal;
isolate the voice signal from the environmental signal;
modify the voice signal and/or the environmental signal; and
generate, via at least one audio output device arranged on, in, or in proximity to the wearable audio device, an audio output signal, wherein the audio output signal comprises a mixed audio signal comprising i) the modified voice signal with the environmental signal; ii) the voice signal with the modified environmental signal; or iii) the modified voice signal with the modified environmental signal.

19. The computer program product of claim 18 , wherein a total time period between the receipt of the audio signal from the at least one microphone to the generation of the audio output signal is less than or equal to 100 milliseconds.

Assignees

• Bose Corp

Inventors

• Laberge Francois
• Stein Charles
• Cowles Colin
• Izarra Felix
• Sisolak Daniel
• Freeman Eric J
• Wax Aric J

Classifications

• H04R1/1083

  Reduction of ambient noise (active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06) · CPC title

• H04R2201/107

  Monophonic and stereophonic headphones with microphone for two-way hands free communication · CPC title

• G10L21/0272

  Voice signal separating · CPC title

• G10K11/17885

  additionally using a desired external signal, e.g. pass-through audio such as music or speech · CPC title

• H04R5/04

  Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} · CPC title