Head-wearable apparatus to generate binaural audio

Patent metadata

Field	Value
Publication number	US-12167218-B2
Application number	US-202318296804-A
Country	US
Kind code	B2
Filing date	Apr 6, 2023
Priority date	Mar 29, 2019
Publication date	Dec 10, 2024
Grant date	Dec 10, 2024

Abstract

Head-wearable apparatus to generate binaural audio content includes a first stem coupled to a first microphone housing that encases first front microphone and first rear microphone that generates acoustic signals, respectively. First microphone housing includes a first front port that faces downward and a first rear port that faces backwards. Apparatus includes second stem coupled to second microphone housing that encases second front microphone and second rear microphone that generate acoustic signals, respectively. Second microphone housing includes second front port that faces downward and second rear port that faces backwards. Apparatus includes binaural audio processor that includes beamformer and storage device. Beamformer generate first beamformer signal based on acoustic signals from first front microphone and first rear microphone, and second beamformer based on acoustic signals from second front microphone and second rear microphone. Storage device stores first and second beamformer signals as a two-channel file.

First claim

What is claimed is:

1. A head-wearable apparatus comprising:
a frame that carries a pair of lenses;
a first stem coupled to a first side of the frame and to a first microphone housing that encases a first front microphone and a first rear microphone that generate acoustic signals;
a second stem coupled to a second side of the frame and to a second microphone housing that encases a second front microphone and a second rear microphone that generate acoustic signals; and
a binaural audio processor that includes
an audio codec to generate
a first decoded acoustic signal based on the acoustic signals from the first front microphone and the first rear microphone, and
a second decoded acoustic signal from the second front microphone and the second rear microphone; and
a beamformer to generate
a first beamformer signal based on the acoustic signals from the first front microphone and the first rear microphone, and
a second beamformer signal based on the acoustic signals from the second front microphone and the second rear microphone, wherein the first beamformer signal and the second beamformer signal are stored as a two-channel file on a storage device.

2. The head-wearable apparatus of claim 1 , wherein the beamformer receives the acoustic signals from the first front microphone, the first rear microphone, the second front microphone and the second rear microphone.

3. The head-wearable apparatus of claim 2 , wherein the head-wearable apparatus is a pair of eyeglasses, wherein the first side of the frame is opposite the second side of the frame.

4. The head-wearable apparatus of claim 3 , wherein the frame carries a pair of lenses and is coupled to a camera, wherein a camera lens of the camera faces forward.

5. The head-wearable apparatus of claim 1 , wherein the beamformer is a fixed beamformer, wherein the fixed beamformer includes a fixed beam pattern that is sub-cardioid or cardioid.

6. The head-wearable apparatus of claim 1 , wherein the first front microphone and the first rear microphone form a first order differential microphone array, and the second front microphone and the second rear microphone form a first order differential microphone array.

7. The head-wearable apparatus of claim 1 ,
wherein the storage device includes a flash storage device.

8. The head-wearable apparatus of claim 1 , wherein the binaural audio processor is a Silicon-on-Chip (SoC).

9. The head-wearable apparatus of claim 8 , further comprising:
a time-division multiplexer (TDM) to process the first decoded acoustic signal and the second decoded acoustic signal and to generate a TDM-processed signal.

10. The head-wearable apparatus of claim 9 , wherein the acoustic signals generated by the first and second front microphones and the first and second rear microphones are pulse density modulation (PDM) signals.

11. The head-wearable apparatus of claim 10 , wherein the first and second decoded acoustic signals are pulse code modulation (PCM) signals.

12. The head-wearable apparatus of claim 11 , wherein the binaural audio processor further comprises:
a noise suppressor to suppress noise from the first beamformer signal and the second beamformer signal and to generate a first noise-suppressed signal and a second noise-suppressed signal.

13. The head-wearable apparatus of claim 12 , wherein the binaural audio processor further comprises:
a speech enhancer to enhance speech from the first noise-suppressed signal and the second noise-suppressed signal to generate a first clean signal and a second clean signal, wherein the storage device stores the first clean signal and the second clean signal.

14. A pair of eyeglasses comprising:
a frame to carry a pair of lenses;
a camera coupled to the frame;
a first stem coupled to a first side of the frame and to a first microphone housing that encases a first front microphone and a first rear microphone that generate acoustic signals;
a second stem coupled to a second side of the frame and to a second microphone housing that encases a second front microphone and a second rear microphone that generate acoustic signals;
a binaural audio processor that includes
an audio codec to generate
a first decoded acoustic signal based on the acoustic signals from the first front microphone and the first rear microphone, and
a second decoded acoustic signal from the second front microphone and the second rear microphone; and
a beamformer to generate
a first beamformer signal based on the acoustic signals from the first front microphone and the first rear microphone, and
a second beamformer signal based on the acoustic signals from the second front microphone and the second rear microphone,
wherein the first beamformer signal and the second beamformer signal are stored as a two-channel file on a storage device.

15. The pair of eyeglasses of claim 14 , wherein the binaural audio processor further comprises:
a noise suppressor to suppress noise from the first beamformer signal and the second beamformer signal and to generate a first noise-suppressed signal and a second noise-suppressed signal.

16. The pair of eyeglasses of claim 15 , wherein the binaural audio processor further comprises:
a speech enhancer to enhance speech from the first noise-suppressed signal and the second noise-suppressed signal to generate a first clean signal and a second clean signal, wherein the storage device stores the first clean signal and the second clean signal.

17. The pair of eyeglasses of claim 14 , wherein the beamformer is a fixed beamformer, wherein the fixed beamformer includes a fixed beam pattern that is sub-cardioid or cardioid.

18. The pair of eyeglasses of claim 14 , further comprising:
a time-division multiplexer (TDM) to process the first decoded acoustic signal and the second decoded acoustic signal and to generate a TDM-processed signal.

19. A method of generating binaural audio content using a head-wearable apparatus comprising:
generating acoustic signals by a first front microphone and a first rear microphone, wherein the first front microphone and the first rear microphone are encased in a first microphone housing that is coupled to a first stem of the head-wearable apparatus;
generating acoustic signals by a second front microphone and a second rear microphone, wherein the second front microphone and the second rear microphone are encased in a second microphone housing that is coupled to a second stem of the head-wearable apparatus, wherein the first stem and the second stem are coupled to opposite sides of a frame of the head-wearable apparatus;
decoding, by a first audio codec, the acoustic signals from the first front microphone and the first rear microphone to generate a first decoded acoustic signal;
decoding, by a second audio codec, the acoustic signals from the second front microphone and the second rear microphone to generate a second decoded acoustic signal;
beamforming, by a beamformer, the acoustic signals from the first front microphone and the first rear microphone to generate a first beamformer signal; and
beamforming, by the beamformer, the acoustic signals from the second front microphone and the second rear microphone to generate a second beamformer signal, wherein the first beamformer signal and the second beamformer signal are stored as a two-channel file on a storage device.

20. The method of claim 19 , wherein
the first decoded acoustic signal is a pulse code modulation (PCM) signal based on the acoustic signals from the first front microphone and
the

Assignees

• Snap Inc

Inventors

• Asfaw Michael
• Hu Dunxu
• Simons Patrick Timothy Mcsweeney
• Wang Limketkai Victoria Lulu

Classifications

• H04R2201/10

  Details of earpieces, attachments therefor, earphones or monophonic headphones covered by H04R1/10 but not provided for in any of its subgroups · CPC title

• H04R1/10 Primary

  Earpieces; Attachments therefor {; Earphones; Monophonic headphones (H04R1/28 takes precedence; stereophonic headphones H04R5/033)} · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• H04R1/406

  microphones · CPC title

• H04R3/005 Primary

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title