Seamless listen-through for a wearable device

Patent metadata

Field	Value
Publication number	US-10951975-B2
Application number	US-202016896010-A
Country	US
Kind code	B2
Filing date	Jun 8, 2020
Priority date	Feb 26, 2019
Publication date	Mar 16, 2021
Grant date	Mar 16, 2021

Abstract

Methods, systems, and devices for signal processing are described. Generally, in one example as provided for by the described techniques, a wearable device includes a processor configured to retrieve a plurality of external microphone signals that includes audio sound from outside of the device from a memory; to separate, based on at least information from an internal microphone signal, a self-voice component from a background component; to perform a first listen-through operation on the separated self-voice component to produce a first listen-through signal; and to produce an output audio signal that is based on at least the first listen-through signal, wherein the output audio signal includes an audio zoom signal that includes audio sound of the plurality of external microphone signals.

First claim

What is claimed is:

1. A wearable device, the wearable device comprising:
a memory configured to store a plurality of external microphone signals that includes audio sound from outside of the device, the audio sound of the plurality of external microphone signals including a self-voice component and a background component; and
a processor configured to retrieve the plurality of external microphone signals that includes audio sound from outside of the device from the memory and to:
separate, based on at least information from an internal microphone signal, the self-voice component of the audio sound of the plurality of external microphone signals from the background component of the audio sound of the plurality of external microphone signals;
perform a first listen-through operation on the separated self-voice component of the audio sound of the plurality of external microphone signals to produce a first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals; and
produce an output audio signal that is based on at least the first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals,
wherein the output audio signal includes an audio zoom signal that includes audio sound of the plurality of external microphone signals.

2. The wearable device of claim 1 wherein the processor is configured to produce the audio zoom signal that includes audio sound of the plurality of external microphone signals by focusing sound pickup in a desired direction.

3. The wearable device of claim 1 wherein the processor is configured to produce the audio zoom signal that includes audio sound of the plurality of external microphone signals by focusing sound pickup on an individual with whom a user wearing the device is conversing.

4. The wearable device of claim 1 wherein the processor is configured to produce the audio zoom signal that includes audio sound of the plurality of external microphone signals from the plurality of external microphone signals based on a beamforming operation.

5. The wearable device of claim 1 wherein the processor is configured to produce the audio zoom signal by suppressing external signals that do not lie in a targeted direction.

6. The wearable device of claim 1 wherein the processor is configured to produce the audio zoom signal by suppressing the self-voice component of the audio sound of the plurality of external microphone signals.

7. The wearable device of claim 1 wherein the processor is configured to produce the audio zoom signal in response to a manual activation of an audio zoom feature.

8. The wearable device of claim 1 wherein the processor is configured to automatically activate, in response to a detected condition, an audio zoom feature to produce the audio zoom signal.

9. The wearable device of claim 1 wherein the audio zoom signal provides a stereo sensation in a targeted direction.

10. The wearable device of claim 1 wherein the audio zoom signal provides natural sounding listen-through features in a targeted direction.

11. The wearable device of claim 1 wherein the processor is further configured to perform foreground sound processing to produce the audio zoom signal.

12. The wearable device of claim 1 wherein the processor is further configured to perform headphone or earphone equalization to produce the audio zoom signal.

13. The wearable device of claim 1 wherein the processor is further configured to perform active noise cancellation compensation to produce the audio zoom signal.

14. The wearable device of claim 1 wherein at a first time, the output audio signal includes the audio zoom signal that includes audio sound of the plurality of external microphone signals, and
wherein at a second time that is different than the first time, the output audio signal includes a signal that is based on the separated background component of the audio sound of the plurality of external microphone signals.

15. A method of audio signal processing, the method comprising:
receiving a plurality of external microphone signals that includes audio sound from outside of the device, the audio sound of the plurality of external microphone signals including a self-voice component and a background component;
based on at least information from an internal microphone signal, separating the self-voice component of the audio sound of the plurality of external microphone signals from the background component of the audio sound of the plurality of external microphone signals;
performing a first listen-through operation on the separated self-voice component of the audio sound of the plurality of external microphone signals to produce a first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals; and
producing an output audio signal that is based on at least the first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals,
wherein the output audio signal includes an audio zoom signal that includes audio sound of the plurality of external microphone signals.

16. A wearable device, the wearable device comprising:
a memory configured to store a plurality of external microphone signals that includes audio sound from outside of the device, the audio sound of the plurality of external microphone signals including a self-voice component and a background component; and
a processor configured to retrieve the plurality of external microphone signals that includes audio sound from outside of the device from the memory and to:
separate, based on at least information from an internal microphone signal, the self-voice component of the audio sound of the plurality of external microphone signals from the background component of the audio sound of the plurality of external microphone signals;
perform a first listen-through operation on the separated self-voice component of the audio sound of the plurality of external microphone signals to produce a first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals; and
produce an output audio signal that is based on at least the first listen-through signal that is based on the separated self-voice component of the audio sound of the plurality of external microphone signals,
wherein the output audio signal includes a signal that is based on the separated background component of the audio sound of the plurality of external microphone signals.

17. The wearable device of claim 16 wherein the processor is configured to separate the self-voice component of the audio sound of the plurality of external microphone signals from the background component of the audio sound of the plurality of external microphone signals using at least one of a multi-microphone speech generative network (MSGN) method or a generalized eigenvalue (GEN) beamforming procedure.

18. The wearable device of claim 16 wherein the processor is further configured to perform a second listen-through operation on the separated background component of the audio sound of the plurality of external microphone signals to produce a second listen-through signal that is based on the separated background component of the audio sound of the plurality of external microphone signals,
wherein the signal that is based on the separated background compo

Assignees

• Qualcomm Inc

Inventors

• Kim Lae-Hoon
• Wang Dongmei
• Saki Fatemeh
• Shahbazi Mirzahasanloo Taher
• Visser Erik
• Alves Rogerio Guedes

Classifications

• H04R2460/01

  Hearing devices using active noise cancellation · CPC title

• H04R2420/07

  Applications of wireless loudspeakers or wireless microphones · CPC title

• H04R2460/13

  Hearing devices using bone conduction transducers · CPC title

• H04R1/1075

  Mountings of transducers in earphones or headphones · CPC title

• H04R1/406

  microphones · CPC title