Device with speaker and image sensor

Patent metadata

Field	Value
Publication number	US-12393396-B2
Application number	US-202318211515-A
Country	US
Kind code	B2
Filing date	Jun 19, 2023
Priority date	Jun 21, 2022
Publication date	Aug 19, 2025
Grant date	Aug 19, 2025

Abstract

In one implementation, a method of playing audio data is performed at a device including a frame configured for insertion into an outer ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, one or more images of a physical environment. The method includes generating audio data based on the one or more images of the physical environment. The method includes playing, via the speaker, the audio data.

First claim

What is claimed is:

1. A method comprising:
at a device including a frame configured for insertion into an outer ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and non-transitory memory:
capturing, using the image sensor, one or more images of a physical environment;
generating audio data based on the one or more images of the physical environment, wherein the image sensor has a device field-of-view different than a user field-of-view and the audio data is based on portions of the one or more images of the physical environment outside the user field-of-view; and
playing, via the speaker, the audio data.

2. The method of claim 1 , wherein generating the audio data based on the one or more images of the physical environment includes transmitting, to a peripheral device, the one or more images of the physical environment and receiving, from the peripheral device, the audio data.

3. The method of claim 1 , wherein generating the audio data includes creating an audio signal.

4. The method of claim 1 , wherein generating the audio data includes altering an audio stream.

5. The method of claim 1 , wherein the device further includes a microphone configured to generate ambient sound data and wherein generating the audio data is further based on the ambient sound data.

6. The method of claim 5 , wherein the ambient sound data includes a vocal input.

7. The method of claim 1 , wherein the device further includes a microphone configured to generate ambient sound data and wherein generating the audio data is independent of the ambient sound data.

8. The method of claim 1 , wherein the device further includes an inertial measurement unit (IMU) configured to generate pose data and wherein generating the audio data is further based on the pose data.

9. The method of claim 1 , wherein the audio data is played spatially from a location based on the one or more images of the physical environment.

10. A device comprising:
a frame configured for insertion into an outer ear;
one or more processors coupled to the frame;
a speaker coupled to the frame and configured to output sound based on audio data received from the one or more processors; and
an image sensor coupled to the frame and configured to provide one or more images of a physical environment to the one or more processors,
wherein the one or more processors are configured to generate the audio data based on the one or more images of the physical environment, and wherein the image sensor has a device field-of-view different than a user field-of-view and the audio data is based on portions of the one or more images of the physical environment outside the user field-of-view.

11. The device of claim 10 , wherein the one or more processors are configured to generate the audio data based on the one or more images of the physical environment by transmitting, to a peripheral device, the one or more images of the physical environment and receiving, from the peripheral device, the audio data.

12. The device of claim 10 , further comprising a microphone configured to generate ambient sound data, wherein the one or more processors are configured to generate the audio data further based on the ambient sound data.

13. The device of claim 10 , further comprising a microphone configured to generate ambient sound data, wherein the one or more processors are configured to generate the audio data independent of the ambient sound data.

14. The device of claim 10 , further comprising an inertial measurement unit (IMU) configured to generate pose data, wherein the one or more processors are configured to generate the audio data further based on the pose data.

15. The device of claim 10 , wherein the audio data is played spatially from a location based on the one or more images of the physical environment.

16. The device of claim 10 , wherein the image sensor includes a fisheye lens.

17. The device of claim 10 , wherein the frame is not physically coupled to a display.

18. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device including a frame configured for insertion into an outer ear, a speaker coupled to the frame, and an image sensor coupled to the frame cause the device to:
capture, using the image sensor, one or more images of a physical environment;
generate audio data based on the one or more images of the physical environment, wherein the image sensor has a device field-of-view different than a user field-of-view and the audio data is based on portions of the one or more images of the physical environment outside the user field-of-view; and
play, via the speaker, the audio data.

Assignees

• Apple Inc

Inventors

• Miller Brett D
• Boothe Daniel K
• Johnson Martin E

Classifications

• H04N7/183

  for receiving images from a single remote source · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F1/1688

  the I/O peripheral being integrated loudspeakers · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title