Seamless listen-through for a wearable device

Patent metadata

Field	Value
Publication number	US-2021204053-A1
Application number	US-202117201998-A
Country	US
Kind code	A1
Filing date	Mar 15, 2021
Priority date	Feb 26, 2019
Publication date	Jul 1, 2021
Grant date	—

Abstract

Methods, systems, and devices for signal processing are described. Generally, as provided for by the described techniques, a wearable device may receive an input audio signal (e.g., including both an external signal and a self-voice signal). The wearable device may detect the self-voice signal in the input audio signal based on a self-voice activity detection (SVAD) procedure, and may implement the described techniques based thereon. The wearable device may perform beamforming operations or other separation procedures to isolate the external signal and the self-voice signal from the input audio signal. The wearable device may apply a first filter to the external signal, and a second filter to the self-voice signal. The wearable device may then mix the filtered signals, and generate an output signal that sounds natural to the user.

First claim

What is claimed is:

1 . A wearable device, the wearable device comprising:
a memory configured to store a plurality of external microphone signals that includes audio sound from outside of the device, the audio sound of the plurality of external microphone signals including a self-voice component and a background component; and a processor configured to retrieve the plurality of external microphone signals that includes audio sound from outside of the device from the memory and to: separate, based on at least information from an internal microphone signal, the self-voice component of the audio sound of the plurality of external microphone signals from the background component of the audio sound of the plurality of external microphone signals; perform a first listen-through operation on the separated self-voice component of the audio sound of the plurality of external microphone signals to produce a first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals; and produce an output audio signal that is based on at least the first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals, wherein the output audio signal includes audio sound based on a beamforming operation.

2 . The wearable device of claim 1 wherein the processor is configured to produce a audio zoom signal that includes audio sound of the plurality of external microphone signals from the plurality of external microphone signals based on the beamforming operation.

3 . The wearable device of claim 2 wherein the processor is configured to produce the audio zoom signal that includes audio sound of the plurality of external microphone signals by focusing sound pickup in a desired direction.

4 . The wearable device of claim 2 wherein the processor is configured to produce the audio zoom signal that includes audio sound of the plurality of external microphone signals by focusing sound pickup on an individual with whom a user wearing the device is conversing.

5 . The wearable device of claim 2 wherein the processor is configured to produce the audio zoom signal by suppressing external signals that do not lie in a targeted direction.

6 . The wearable device of claim 2 wherein the processor is configured to produce the audio zoom signal by suppressing the self-voice component of the audio sound of the plurality of external microphone signals.

7 . The wearable device of claim 2 wherein the processor is configured to produce the audio zoom signal in response to a manual activation of an audio zoom feature.

8 . The wearable device of claim 2 wherein the audio zoom signal provides a stereo sensation in a targeted direction.

9 . The wearable device of claim 2 wherein the audio zoom signal provides natural sounding listen-through features in a targeted direction.

10 . The wearable device of claim 2 wherein the processor is further configured to perform foreground sound processing to produce the audio zoom signal.

11 . The wearable device of claim 2 wherein the processor is further configured to perform headphone or earphone equalization to produce the audio zoom signal.

12 . The wearable device of claim 2 wherein the processor is further configured to perform active noise cancellation compensation to produce the audio zoom signal.

13 . The wearable device of claim 2 wherein at a first time, the output audio signal includes the audio zoom signal that includes audio sound of the plurality of external microphone signals, and
wherein at a second time that is different than the first time, the output audio signal includes a signal that is based on the separated background component of the audio sound of the plurality of external microphone signals.

14 . The wearable device of claim 1 wherein the processor is configured to automatically activate, in response to a detected condition, an audio zoom feature to produce an audio zoom signal.

15 . The wearable device of claim 1 wherein the output audio signal includes a signal that is based on the separated background component of the audio sound of the plurality of external microphone signals.

16 . The wearable device of claim 15 wherein the processor is further configured to perform a second listen-through operation on the separated background component of the audio sound of the plurality of external microphone signals to produce a second listen-through signal that is based on the separated background component of the audio sound of the plurality of external microphone signals,
wherein the signal that is based on the separated background component of the audio sound of the plurality of external microphone signals includes at least the second listen-through signal that is based on the separated background component of the audio sound of the plurality of external microphone signals.

17 . The wearable device of claim 1 wherein the processor is configured to separate the self-voice component of the audio sound of the plurality of external microphone signals from the background component of the audio sound of the plurality of external microphone signals using at least one of a multi-microphone speech generative network (MSGN) method or a generalized eigenvalue (GEN) beamforming procedure.

18 . The wearable device of claim 1 wherein the plurality of external microphone signals includes a left microphone signal and a right microphone signal.

19 . The wearable device of claim 1 wherein the processor is further configured to perform an active noise cancellation (ANC) operation on at least the internal microphone signal and at least one external microphone signal of the plurality of external microphone signals to produce an ANC signal, and
wherein the output audio signal is based on the ANC signal.

20 . The wearable device of claim 19 wherein the processor is configured to perform the ANC operation in a codec and to separate the self-voice component from the background component in a digital signal processor.

21 . The wearable device of claim 1 wherein the processor is configured to separate the self-voice component from the background component based on at least a difference between a phase of the internal microphone signal and a phase of at least one external microphone signal of the plurality of external microphone signals.

22 . The wearable device of claim 1 wherein the device further comprises a bone conduction microphone, and wherein the separated self-voice component of the audio sound of the plurality of external microphone signals is based on an output of the bone conduction microphone.

23 . The wearable device of claim 1 wherein the device further comprises an external microphone arranged to receive an acoustic signal from an ambient environment, wherein a corresponding one of the plurality of external microphone signals is based on an output of the external microphone.

24 . The wearable device of claim 1 wherein the device further comprises an internal microphone arranged to receive an acoustic signal from within an ear canal, wherein the internal microphone signal is based on an output of the internal microphone.

25 . The wearable device of claim 1 wherein the device further comprises a loudspeaker configured to produce a first acoustic signal based on the output audio signal.

Assignees

• Qualcomm Inc

Inventors

• Kim Lae-Hoon
• Wang Dongmei
• Saki Fatemeh
• Shahbazi Mirzahasanloo Taher
• Visser Erik
• Alves Rogerio Guedes

Classifications

• H04R1/1083 Primary

  Reduction of ambient noise (active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06) · CPC title

• H04R2460/13

  Hearing devices using bone conduction transducers · CPC title

• H04R2460/01

  Hearing devices using active noise cancellation · CPC title

• H04R2420/07

  Applications of wireless loudspeakers or wireless microphones · CPC title

• H04R1/1075

  Mountings of transducers in earphones or headphones · CPC title