Camera control using system sensor data

Patent metadata

Field	Value
Publication number	US-12101549-B2
Application number	US-202217856760-A
Country	US
Kind code	B2
Filing date	Jul 1, 2022
Priority date	Jul 7, 2021
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

A method for using cameras in an augmented reality headset is provided. The method includes receiving a signal from a sensor mounted on a headset worn by a user, the signal being indicative of a user intention for capturing an image. The method also includes identifying the user intention for capturing the image, based on a model to classify the signal from the sensor according to the user intention, selecting a first image capturing device in the headset based on a specification of the first image capturing device and the user intention for capturing the image, and capturing the image with the first image capturing device. An augmented reality headset, a memory storing instructions, and a processor to execute the instructions to cause the augmented reality headset as above are also provided.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving a signal from a sensor mounted on a headset worn by a user, the signal being indicative of a user intention for capturing an image;
receiving an indication of an orientation of the headset;
identifying the user intention for capturing the image, based on a model to classify the signal from the sensor according to the user intention;
selecting a first image capturing device in the headset based on a specification of the first image capturing device and the user intention for capturing the image, wherein a field of view of the first image capturing device is aligned with the orientation of the headset; and
capturing the image with the first image capturing device.

2. The computer-implemented method of claim 1 , wherein receiving the signal from the sensor comprises one of receiving an inertial signal from an inertial motion sensor, receiving a sound capture from a user voice, receiving a hand gesture, or receiving an active button press.

3. The computer-implemented method of claim 1 , wherein selecting the first image capturing device based on the specification of the first image capturing device comprises selecting the first image capturing device when the field of view of the first image capturing device includes a point of interest in a field of view for the user, in the headset.

4. The computer-implemented method of claim 1 , further comprising de-activating the first image capturing device in the headset when the user intention is incompatible with the specification of the first image capturing device.

5. The computer-implemented method of claim 1 , further comprising selecting, from one or more image capturing devices in the headset, an image capturing device whose specification best matches the user intention.

6. The computer-implemented method of claim 1 , further comprising selecting a second image capturing device and de-activating the first image capturing device based on a second user intention.

7. The computer-implemented method of claim 1 , wherein the specification of the first image capturing device is the field of view, and wherein selecting the first image capturing device comprises verifying that the field of view includes an object of interest identified within the user intention.

8. The computer-implemented method of claim 1 , wherein receiving the signal from the sensor mounted on the headset comprises identifying a hand gesture of the user indicative of an object of interest.

9. The computer-implemented method of claim 1 , wherein receiving the signal from the sensor comprises receiving a pupil position of the user from an eye tracking device mounted on the headset.

10. The computer-implemented method of claim 1 , wherein receiving the indication of hte orientation of the headset comprises identifying the orientation of the headset, and selecting the first image capturing device when the field of view is aligned with the orientation of the headset.

11. An augmented reality headset, comprising:
a first camera and a second camera mounted on a frame, having a first field of view, and a second field of view, respectively;
a sensor mounted on the frame;
a memory configured to store multiple instructions; and
one or more processors configured to execute the instructions to cause the augmented reality headset to:
receive a signal from the sensor mounted on the frame, the signal being indicative of an intention of a user for capturing an image,

receive an indication of an orientation of the augmented reality headset,
identify the intention of the user for capturing the image, based on a model to classify the signal from the sensor according to the intention of the user,
select one of the first camera or the second camera based on the first field of view, the second field of view, and the intention of the user for capturing the image, wherein a field of view of a selected camera is aligned with the orientation of the augmented reality headset, and
capture the image with the selected camera.

12. The augmented reality headset of claim 11 , wherein the sensor is an inertial motion unit, and to receive the signal from the sensor comprises identifying the orientation of the augmented reality headset relative to a fixed coordinate system.

13. The augmented reality headset of claim 11 , wherein to select one of the first camera or the second camera the one or more processors execute instructions to select the first camera when the first field of view of the first camera includes a point of interest in a field of view for the user, in the augmented reality headset.

14. The augmented reality headset of claim 11 , wherein the one or more processors execute instructions to de-activate the first camera when the second camera is selected, based on a second user intention.

15. The augmented reality headset of claim 11 , wherein the sensor provides the signal indicative of a hand gesture of the user pointing to an object of interest.

16. The augmented reality headset of claim 11 , wherein the sensor is an eye tracking device mounted on the frame, configured to provide the signal indicative of a pupil position of the user.

17. The augmented reality headset of claim 11 , wherein the sensor is an inertial motion sensor configured to provide the signal indicative of the orientation of the augmented reality headset.

18. The augmented reality headset of claim 11 , wherein the sensor is one of the first camera or the second camera, configured to capture a gesture of the user indicative of an object of interest.

19. The augmented reality headset of claim 11 , wherein the sensor is a microphone configured to collect and identify a voice command indicative of the intention of the user.

20. The augmented reality headset of claim 11 , wherein the sensor is a touch-sensitive sensor configured to receive a touch command from the user.

Assignees

• Meta Platforms Tech Llc

Inventors

• Shroff Sapna
• Bie Linsen
• Hu Jun
• Tao David
• Karanikos Demetrios Basil
• Sztuk Sebastian
• Zhang Zhaonian
• Kuang Jiangtao
• Luo Danni
• Li Yilei

Classifications

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/90 Primary

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title