Gesture-based application invocation

Patent metadata

Field	Value
Publication number	US-12265663-B2
Application number	US-202217657911-A
Country	US
Kind code	B2
Filing date	Apr 4, 2022
Priority date	Apr 4, 2022
Publication date	Apr 1, 2025
Grant date	Apr 1, 2025

Abstract

A head-worn device system includes one or more cameras, one or more display devices and one or more processors. The system also includes a memory storing instructions that, when executed by the one or more processors, configure the system to detect a gesture made by a user of the computing apparatus and generate gesture data identifying the gesture, select an application or selected action from a set of registered applications and actions based on the gesture data, and invoke the application or selected action.

First claim

What is claimed is:

1. A computer-implemented method comprising:
storing, by one or more processors of an Augmented Reality (AR) device, a plurality of registration tags, each registration tag associating a gesture to a registered application and a first action;
detecting, by the one or more processors, using one or more cameras of the AR device, an unprompted gesture made by a user of the AR device;
mapping, by the one or more processors, the unprompted gesture to a plurality of registered applications using the unprompted gesture and the plurality of registration tags by matching the unprompted gesture to the gesture of two or more of the plurality of registration tags; and
in response to the matching of the unprompted gesture to the gesture of two or more of the plurality of registration tags, performing operations comprising:
providing a user interface having a display of the plurality of registered applications selectable by the user;
determining a user selection of a selected application from among the plurality of registered applications using the user interface;
determining a second action using the selected application and a respective registration tag of the plurality of registration tags;
and
invoking, by the one or more processors, the selected application, to cause the selected application to be loaded into a working memory and executed by the one or more processors to perform the second action.

2. The computer-implemented method of claim 1 , wherein determining the user selection of the selected application further comprises:
determining that the selected application is a preset default application associated with the detected gesture.

3. The computer-implemented method of claim 1 , wherein the unprompted gesture is a contactless gesture.

4. The computer-implemented method of claim 1 , wherein the AR device is glasses.

5. The computer-implemented method of claim 1 , wherein invoking the selected application further comprises passing input parameters to the selected application.

6. A computing apparatus comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, configure the computing apparatus to perform operations comprising:
storing, by one or more processors of an Augmented Reality (AR) device, a plurality of registration tags, each registration tag associating a gesture to a registered application and a first action;
detecting, by the one or more processors, using one or more cameras of the AR device, an unprompted gesture made by a user of the AR device;
mapping, by the one or more processors, the unprompted gesture to a plurality of registered applications using the unprompted gesture and the plurality of registration tags by matching the unprompted gesture to the gesture of two or more of the plurality of registration tags; and
in response to the matching of the unprompted gesture to the gesture of two or more of the plurality of registration tags, performing operations comprising:
providing a user interface having a display of the plurality of registered applications selectable by the user;
determining a user selection of a selected application from among the plurality of registered applications using the user interface;
determining a second action using the selected application and a respective registration tag of the plurality of registration tags;
and
invoking, by the one or more processors, the selected application, to cause the selected application to be loaded into a working memory and executed by the one or more processors to perform the second action.

7. The computing apparatus of claim 6 , wherein determining the user selection of the selected application further comprises:
determining that the selected application is a preset default application associated with the detected gesture.

8. The computing apparatus of claim 6 , wherein the unprompted gesture is a contactless gesture.

9. The computing apparatus of claim 6 , wherein the AR device is glasses.

10. The computing apparatus of claim 6 , wherein invoking the selected application further comprises passing input parameters to the selected application.

11. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:
storing, by one or more processors of an Augmented Reality (AR) device, a plurality of registration tags, each registration tag associating a gesture to a registered application and a first action;
detecting, by the one or more processors, using one or more cameras of the AR device, an unprompted gesture made by a user of the AR device;
mapping, by the one or more processors, the unprompted gesture to a plurality of registered applications using the unprompted gesture and the plurality of registration tags by matching the unprompted gesture to the gesture of two or more of the plurality of registration tags; and
in response to the matching of the unprompted gesture to the gesture of two or more of the plurality of registration tags, performing operations comprising:
providing a user interface having a display of the plurality of registered applications selectable by the user;
determining a user selection of a selected application from among the plurality of registered applications using the user interface;
determining a second action using the selected application and the plurality of registration tags;
and
invoking, by the one or more processors, the selected application, to cause the selected application to be loaded into a working memory and executed by the one or more processors to perform the second action.

12. The non-transitory computer-readable storage medium of claim 11 , wherein determining the user selection of the selected application further comprises:
determining that the selected application is a preset default application associated with the detected gesture.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the unprompted gesture is a contactless gesture.

14. The non-transitory computer-readable storage medium of claim 11 , wherein the AR device is glasses.

15. The non-transitory computer-readable storage medium of claim 11 , wherein invoking the selected application further comprises passing input parameters to the selected application.

Assignees

• Snap Inc

Inventors

• Moll Sharon
• Gurgul Piotr
• Sullivan Francis Patrick
• Rybin Andrei

Classifications

• G06F3/0482 Primary

  Interaction with lists of selectable items, e.g. menus · CPC title

• G02B27/017

  Head mounted · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F1/1686

  the I/O peripheral being an integrated camera · CPC title