Single-handed microgesture inputs

Patent metadata

Field	Value
Publication number	US-11656689-B2
Application number	US-202217647941-A
Country	US
Kind code	B2
Filing date	Jan 13, 2022
Priority date	Nov 30, 2020
Publication date	May 23, 2023
Grant date	May 23, 2023

Abstract

A method for single-handed microgesture input comprises receiving hand tracking data for a hand of a user. A set of microgesture targets that include software functions are assigned to positions along a length of a first finger. The received hand tracking data is analyzed by a gesture recognition machine. A location of a thumbtip of the hand of the user is determined relative to the positions along the first finger. Responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, a corresponding first microgesture target is designated for selection. Selection of the first microgesture target is enabled based on a duration the thumbtip is at the first position. Responsive to detecting a confirmation action, the corresponding microgesture target executes.

First claim

The invention claimed is:

1. A method for single-handed microgesture input, comprising:
receiving hand tracking data for a hand of a user;
assigning a set of microgesture targets to a plurality of positions along a length of a first finger of the hand of the user, each microgesture target including one or more software functions;
based on an output of a gesture recognition machine configured to analyze the received hand tracking data:
determining a location of a thumbtip of the hand of the user relative to the plurality of positions along the length of the first finger;
responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, designating a first microgesture target for selection, the first microgesture target corresponding to such first position; and
enabling selection of the first microgesture target based on a duration the thumbtip is at the first position; and

responsive to detecting a confirmation action, executing the corresponding first microgesture target.

2. The method of claim 1 , further comprising: providing a visual affordance to the user, the visual affordance including indicators for one or more assigned microgesture targets, the visual affordance presented within a field-of-view of the user at a display location not dependent on a location of the hand of the user.

3. The method of claim 2 , wherein the display location of the visual affordance is based on a gaze direction of the user.

4. The method of claim 2 , wherein the visual affordance is body-locked to the user.

5. The method of claim 2 , wherein two or more indicators for the microgesture targets are presented in an order based on corresponding positions along the length of the first finger.

6. The method of claim 1 , further comprising:
activating single-handed microgesture input responsive to recognizing an activation command from the user.

7. The method of claim 1 , further comprising:
responsive to determining that the thumbtip has moved to a second position along the length of the first finger, designating a corresponding second microgesture target for selection; and
enabling selection of the second microgesture target based on a duration the thumbtip is within a threshold distance of the second position.

8. The method of claim 7 , wherein the gesture recognition machine is configured to determine a position of the thumbtip along a continuum of positions along the length of the first finger.

9. The method of claim 8 , wherein the continuum of positions along the length of the first finger are divided into to a plurality of discrete positions for microgesture target assignment.

10. The method of claim 2 , wherein the visual affordance displays a subset of the indicators for the assigned microgesture targets, the displayed subset of indicators based on the location of the thumbtip of the hand of the user relative to the plurality of positions along the length of the first finger.

11. The method of claim 1 , wherein the confirmation action includes a dwell time at the first position greater than a threshold.

12. The method of claim 1 , wherein the confirmation action includes a predetermined gesture.

13. The method of claim 1 , wherein sets of microgesture targets are assigned to pluralities of positions along a length of two or more fingers of the hand of the user.

14. The method of claim 1 , further comprising:
receiving hand tracking data for a second hand of a user;
assigning a second set of microgesture targets to a plurality of positions along a length of a first finger of the second hand of the user;
based on output of the gesture recognition machine, determining a location of a thumbtip of the second hand of the user relative to the plurality of positions along the length of the first finger of the second hand of the user; and
responsive to detecting a confirmation action, executing the corresponding microgesture target.

15. The method of claim 1 , wherein receiving hand tracking data for the hand of the user includes:
receiving depth data for an environment;
fitting a virtual skeleton to point clouds of the received depth data;
assigning hand joints to the virtual skeleton based at least in part on image data of the user positioning the thumbtip of the hand relative to the first finger of the hand; and
tracking positions of the assigned hand joints across sequential depth images.

16. A system for a head-mounted display, comprising:
one or more outward-facing image sensors;
a display device configured to present virtual content; and
a gesture recognition machine configured to:
receive hand tracking data for a hand of a user via the one or more outward facing image sensors;
assign a set of microgesture targets to a plurality of positions along a length of a first finger of the hand of the user, each microgesture target including one or more software functions;
determine a location of a thumbtip of the hand of the user relative to the plurality of positions along the length of the first finger;
responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, designate a first microgesture target for selection, the first microgesture target corresponding to such first position;
enabling selection of the first microgesture target based on a duration the thumbtip is at the first position; and
responsive to detecting a confirmation action, execute the corresponding microgesture target.

17. The system of claim 16 , wherein the gesture recognition machine is further configured to:
responsive to determining that the thumbtip has moved to a second position along the length of the first finger, augment an indicator for designating a corresponding second microgesture target for selection; and
enabling selection of the second microgesture target based on a duration the thumbtip is within a threshold distance of the second position.

18. The system of claim 16 , wherein the gesture recognition machine is further configured to determine a position of the thumbtip along a continuum of positions along the length of the first finger.

19. The method of claim 18 , wherein the continuum of positions along the length of the first finger are divided into to a plurality of discrete positions for microgesture target assignment.

20. A method for single-handed microgesture input, comprising:
receiving hand tracking data for a hand of a user;
assigning a set of microgesture targets to a plurality of positions along a length of a first finger of the hand of the user;
providing a visual affordance to the user, the visual affordance including indicators for two or more assigned microgesture targets, the visual affordance presented within a field-of-view of the user at a position not dependent on a position of the hand of the user; and
based on an output of a gesture recognition machine configured to analyze the received hand tracking data:
determining a location of a thumbtip of the hand of the user along a continuum of positions along the length of the first finger, the continuum of positions along the length of the first finger divided into to a plurality of discrete positions for microgesture target assignment;
responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, designating a first microgesture target for selection, the first microgestur

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Schwarz Julia
• Barragan Noe Moreno
• Notter Michael Harley
• Tang sheng kai
• Neff Joshua Kyle

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title