Hand-tracking pipeline dimming

Patent metadata

Field	Value
Publication number	US-12382188-B2
Application number	US-202217947947-A
Country	US
Kind code	B2
Filing date	Sep 19, 2022
Priority date	Jun 22, 2022
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

A hand-tracking input pipeline dimming system for an AR system is provided. The AR system deactivates the hand-tracking input pipeline and places a camera component of the hand-tracking input pipeline in a limited operational mode. The AR system uses the camera component to detect initiation of a gesture by a user of the AR system and in response to detecting the initiation of the gesture, the AR system activates the hand-tracking input pipeline and places the camera component in a fully operational mode.

First claim

What is claimed is:

1. A computer-implemented method comprising:
deactivating, by one or more processors of an Augmented Reality (AR) system, a hand-tracking input pipeline of the AR system;
instructing, by the one or more processors, a camera component of the AR system to enter into a limited operational mode, the limited operational mode allowing capturing limited frame rate video data;
capturing, by the one or more processors using the camera component, the limited frame rate video data;
detecting, by the one or more processors using the limited frame rate video data, initiation of a gesture by a user of the AR system; and
performing, by the one or more processors, on a basis of detecting the initiation of the gesture, operations comprising:
activating the hand-tracking input pipeline while the user is still making the gesture of the detected initiation; and
instructing the camera component to enter into a fully operational mode to recognize the gesture while the user is still making the gesture.

2. The method of claim 1 , wherein detecting the initiation of the gesture further comprises:
recognizing the initiation of the gesture using a binary gesture classifier.

3. The method of claim 1 , wherein the limited operational mode of the camera component includes a limited operational frame rate.

4. The method of claim 3 , wherein the limited operational frame rate is a frame rate less than a fully operational frame rate.

5. The method of claim 1 , wherein
the camera component comprises a plurality of cameras, and
wherein instructing the camera component of the AR system to enter into a limited operational mode further comprises:
instructing the camera component to selectively turn off one or more cameras of the camera component.

6. The method of claim 1 , further comprising:
on a basis of detecting the initiation of the gesture, performing operations comprising:
setting a timer; and
on a basis of determining that the timer has expired, performing operations comprising;
deactivating the one or more components of the hand-tracking input pipeline; and
instructing the camera component to enter into the limited operational mode.

7. The method of claim 1 , wherein the AR system comprises a head-worn device.

8. A computing apparatus comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations comprising:
deactivating a hand-tracking input pipeline of an AR system;
instructing a camera component of the AR system to enter into a limited operational mode, the limited operational mode allowing capturing limited frame rate video data;
capturing, using the camera component, the limited frame rate video data;
detecting, using the limited frame rate video data, initiation of a gesture by a user of the AR system; and
performing, on a basis of detecting the initiation of the gesture, operations comprising:
activating the hand-tracking input pipeline while the user is still making the gesture of the detected initiation; and
instructing the camera component to enter into a fully operational mode to recognize the gesture while the user is still making the gesture.

9. The computing apparatus of claim 8 , wherein the instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations of detecting the initiation of the gesture further comprise:
recognizing the initiation of the gesture using a binary gesture classifier.

10. The computing apparatus of claim 8 , wherein the limited operational mode of the camera component includes a limited operational frame rate.

11. The computing apparatus of claim 10 , wherein the limited operational frame rate is a frame rate less than a fully operational frame rate.

12. The computing apparatus of claim 8 ,
wherein the camera component comprises a plurality of cameras, and
wherein the instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising instructing the camera component of the AR system to enter into a limited operational mode further cause the computing system to perform operations comprising:
instructing the camera component to selectively turn off one or more cameras of the camera component.

13. The computing apparatus of claim 8 , wherein the instructions when executed by the one or more processors further cause the computing apparatus to perform operations comprising:
performing, by the one or more processors, on a basis of detecting the initiation of the gesture, operations comprising:
setting, by the one or more processors a timer;
performing, on a basis of determining that the timer has expired, operations comprising;
deactivating one or more components of the hand-tracking input pipeline; and
placing the camera component in a limited operational mode.

14. The computing apparatus of claim 8 , wherein the AR system comprises a head-worn device.

15. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by one or more processors of a computer, cause the computer to perform operations comprising:
deactivating a hand-tracking input pipeline of a AR system in a deactivated mode;
placing a camera component of the AR system in a limited operational mode, the limited operational mode allowing capturing limited frame rate video data;
capturing, using the camera component, the limited frame rate video data;
detecting, using the limited frame rate video data, initiation of a gesture by a user of the AR system; and
performing, on a basis of detecting the initiation of the gesture, operations comprising:
activating the hand-tracking input pipeline while the user is still making the gesture of the detected initiation; and
instructing the camera component to enter into a fully operational mode to recognize the gesture while the user is still making the gesture.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions that when executed by one or more processors of a computer cause the computer to perform operations of detecting the initiation of the gesture further comprise:
recognizing the initiation of the gesture using a binary gesture classifier.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the limited operational mode of the camera component includes a limited operational frame rate.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the limited operational frame rate is a frame rate less than a fully operational frame rate.

19. The non-transitory computer-readable storage medium of claim 15 ,
wherein the camera component comprises a plurality of cameras, and
wherein the instructions that, when executed by the one or more processors, cause the computer to perform operations comprising instructing the camera component of the AR system to enter into a limited operational mode further cause the computer to perform operations comprising:
instructing the camera component to selectively turn off one or more cameras of the camera component.

20. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions when executed by one or more processors of a computer further cause the computer to perform operations comprising:
performing, on a basis of detecting the initiation of the gesture, operations comprising:
activatin

Assignees

• Snap Inc

Inventors

• Bajana Jan
• Colascione Daniel
• Evangelidis Georgios
• Mendez Mendez Erick
• Wolf Daniel

Classifications

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/651

  for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title