Eyewear with direction of sound arrival detection

Patent metadata

Field	Value
Publication number	US-12411362-B2
Application number	US-202117539839-A
Country	US
Kind code	B2
Filing date	Dec 1, 2021
Priority date	Dec 1, 2021
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

Eyewear providing a visual indicator to a hearing-impaired user indicating a direction of arrival of a sound relative to the eyewear to help the user obtain greater awareness of the surrounding environment. An eyewear optical assembly includes an image display displaying the visual indicator discernable by the user and corresponding to the direction of arrival of the sound, even when a sound source is not viewable through the optical assembly. The image display also displays an image indicative of the sound source. A front portion of an eyewear frame includes a light array, where one or more lights of the light array is illuminated to indicate the direction of arrival and intensity of the sound source. An array of vibrating devices may also be used to indicate the direction of arrival and intensity of the sound source.

First claim

What is claimed is:

1. Eyewear, comprising:
a frame having a first side and a second side;
a first temple adjacent the first side of the frame;
an optical element supported by the frame;
a computer vision (CV) camera supported by the frame and configured to perform object identification;
a microphone array supported by the first temple and configured to receive a sound from a remote sound source that generates sound signals;
a vibration device array configured to generate a vibration pattern indicative of the remote sound source; and
an electronic processor configured to:
process the sound signals received from the microphone array to determine a direction of the received sound relative to the eyewear;
access a memory comprising a data set of known sounds correlated to known sound sources to identify the received sound;
generate an indicator discernable by a user of the eyewear that is indicative of the direction of the received sound relative to the eyewear; and
control the vibration device array to generate the vibration pattern indicative of the remote sound source, wherein a variable intensity of the vibration pattern is indicative of a variable intensity of the sound and varies directly with the intensity of the sound;

wherein the processor is configured to identify the sound source using both the CV camera and the microphone array to identify the received sound by comparing the received sound to the memory data, and by correlating the received sound using the CV camera object identification.

2. The eyewear of claim 1 , wherein the indicator indicates a direction of the received sound from the sound source that is outside a view of the optical element.

3. The eyewear of claim 1 , wherein the optical element further comprises a see- through display configured to generate images, and the electronic processor is configured to display an image on the display that is indicative of the sound source as the indicator.

4. The eyewear of claim 1 , further comprising lights supported by the frame, wherein the lights comprise at least one left light positioned by the frame first side and at least one right light positioned by the frame second side, wherein the processor is configured to illuminate at least one of the lights as the indicator.

5. The eyewear of claim 4 , wherein the lights comprise an array of lights that extend from the first side of the frame to the second side of the frame.

6. The eyewear of claim 5 , wherein the lights are positioned on a bridge extending between the frame first side and the frame second side.

7. The eyewear of claim 6 , wherein the processor is configured to illuminate one light of the array of lights at a brightness that corresponds to an intensity of a sound.

8. A method for use with eyewear, the eyewear having a frame, a first temple adjacent a first side of the frame, an optical element supported by the frame, a computer vision (CV) camera supported by the frame and configured to perform object identification, a microphone array supported by the first temple and configured to receive a sound from a remote sound source that generates sound signals, a vibration device array configured to generate a vibration pattern indicative of the remote sound source; and an electronic processor, the processor:
processing the sound signals received from the microphone array to determine a direction of the received sound relative to the eyewear;
accessing a memory comprising a data set of known sounds correlated to known sound sources to identify the received sound;
generating an indicator discernable by a user of the eyewear that is indicative of the direction of the received sound relative to the eyewear; and
controlling the vibration device array to generate the vibration pattern indicative of the remote sound source, wherein a variable intensity of the vibration pattern is indicative of a variable intensity of the sound and varies directly with the intensity of the sound; and
identifying the sound source using both the CV camera and the microphone array to identify the received sound by comparing the received sound to the memory data, and by correlating the received sound using the CV camera object identification.

9. The method of claim 8 , wherein the indicator indicates a direction of received sound from the sound source that is outside a view of the optical element.

10. The method of claim 8 , wherein the optical element further comprises a see-through display configured to generate images, wherein the electronic processor is configured to display an image on the display that is indicative of the sound source as the indicator.

11. The method of claim 8 , wherein the processor illuminates one light of the array of lights at a brightness that corresponds to an intensity of a sound.

12. A non-transitory computer-readable medium storing program code which, when executed, is operative to cause an electronic processor of eyewear having a frame, a first temple adjacent a first side of the frame, a computer vision (CV) camera supported by the frame and configured to perform object identification, an optical element supported by the frame, and a microphone array supported by the first temple and configured to receive a sound from a remote sound source that generates sound signals, a vibration device array configured to generate a vibration pattern indicative of the remote sound source; to perform the steps of:
processing the sound signals received from the microphone array to determine a direction of the received sound relative to the eyewear;
accessing a memory comprising a data set of known sounds correlated to known sound sources to identify the received sound;
generating an indicator discernable by a user of the eyewear that is indicative of the direction of the received sound relative to the eyewear;
controlling the vibration device array to generate the vibration pattern indicative of the remote sound source, wherein a variable intensity of the vibration pattern is indicative of a variable intensity of the sound and varies directly with the intensity of the sound; and
identifying the sound source using both the CV camera and the microphone array to identify the received sound by comparing the received sound to the memory data, and by correlating the received sound using the CV camera object identification.

13. The non-transitory computer-readable medium as specified in claim 12 , wherein the variable intensity of the vibration pattern varies directly with the intensity of the sound.

Assignees

• Pounds Jennica
• Harris Daniel
• Arya Ashwani
• Snap Inc

Inventors

• Pounds Jennica
• Harris Daniel
• Arya Ashwani

Classifications

• G02C11/04

  Illuminating means · CPC title

• G02C5/001

  specially adapted for particular purposes, not otherwise provided for or not fully classifiable according to technical characteristics, e.g. therapeutic glasses · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/0176

  characterised by mechanical features · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title