Velocity field interaction for free space gesture interface and control

Patent metadata

Field	Value
Publication number	US-11726575-B2
Application number	US-202117379915-A
Country	US
Kind code	B2
Filing date	Jul 19, 2021
Priority date	Oct 16, 2013
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

The technology disclosed relates to automatically interpreting a gesture of a control object in a three dimensional sensor space by sensing a movement of the control object in the three dimensional sensor space, sensing orientation of the control object, defining a control plane tangential to a surface of the control object and interpreting the gesture based on whether the movement of the control object is more normal to the control plane or more parallel to the control plane.

First claim

What is claimed is:

1. A method of navigating a multi-layer presentation tree using gestures of a hand, which is a control object in a three-dimensional (3D) sensor space, using a 3D sensor, the method including:
processing an output of a camera of the 3D sensor thereby sensing a movement of the hand in any direction in the 3D sensor space using a control plane that remains tangent to a surface of a palm of the hand or a surface of the back of the hand throughout the movement of the hand, the control plane being defined by a sensed orientation of the surface of the palm of the hand or the surface of the back of the hand, such that an orientation of the control plane adjusts according to the sensed orientation of the surface of the palm of the hand or the surface of the back of the hand; and
interpreting by a computing device a direction of the movement of the hand as scrolling through a particular level of the multi-layer presentation tree when a direction of the movement of the hand is more normal with respect to the surface of the tangent control plane than parallel with respect to the surface of the control plane.

2. The method of claim 1 , wherein the hand includes a detected sub-object that is a portion of the hand and wherein the method further includes processing a sensed movement of the sub-object and interpreting the sensed movement of the sensed sub-object as a gesture interacting with the multi-layer presentation tree.

3. The method of claim 2 , wherein the sub-object is a finger of the hand.

4. The method of claim 1 , further including: (i) traversing menus responsive to movement of a first sub-object of a first hand and (ii) traversing menu paths responsive to movement of a second sub-object of a second hand.

5. A non-transitory computer-readable recording medium having computer instructions recorded thereon, the computer instructions, when executed by one or more processors, cause the one or more processors to perform the method of:
processing an output of a camera of the 3D sensor thereby sensing a movement of the hand in any direction in the 3D sensor space using a control plane that remains tangent to a surface of a palm of the hand or a surface of the back of the hand throughout the movement of the hand, the control plane being defined by a sensed orientation of the surface of the palm of the hand or the surface of the back of the hand, such that an orientation of the control plane adjusts according to the sensed orientation of the surface of the palm of the hand or the surface of the back of the hand; and
interpreting by a computing device a direction of the movement of the hand as scrolling through a particular level of the multi-layer presentation tree when a direction of the movement of the hand is more normal with respect to the surface of the tangent control plane than parallel with respect to the surface of the control plane.

6. The method of claim 5 , wherein the hand includes a detected sub-object that is a portion of the hand and wherein the method further includes processing a sensed movement of the sub-object and interpreting the sensed movement of the sensed sub-object as a gesture interacting with the multi-layer presentation tree.

7. The method of claim 6 , wherein the sub-object is a finger of the hand.

8. The method of claim 5 , further including: (i) traversing menus responsive to movement of a first sub-object of a first hand and (ii) traversing menu paths responsive to movement of a second sub-object of a second hand.

9. A system comprising a memory storing computer instructions and one or more processors, the computer instructions, when executed by the one or more processors, cause the one or more processors to perform the method of:
processing an output of a camera of the 3D sensor thereby sensing a movement of the hand in any direction in the 3D sensor space using a control plane that remains tangent to a surface of a palm of the hand or a surface of the back of the hand throughout the movement of the hand, the control plane being defined by a sensed orientation of the surface of the palm of the hand or the surface of the back of the hand, such that an orientation of the control plane adjusts according to the sensed orientation of the surface of the palm of the hand or the surface of the back of the hand; and
interpreting by a computing device a direction of the movement of the hand as scrolling through a particular level of the multi-layer presentation tree when a direction of the movement of the hand is more normal with respect to the surface of the tangent control plane than parallel with respect to the surface of the control plane.

10. The method of claim 9 , wherein the hand includes a detected sub-object that is a portion of the hand and wherein the method further includes processing a sensed movement of the sub-object and interpreting the sensed movement of the sensed sub-object as a gesture interacting with the multi-layer presentation tree.

11. The method of claim 10 , wherein the sub-object is a finger of the hand.

12. The method of claim 9 , further including: (i) traversing menus responsive to movement of a first sub-object of a first hand and (ii) traversing menu paths responsive to movement of a second sub-object of a second hand.

Assignees

• Ultrahaptics Ip Two Ltd

Inventors

• Cohen Isaac
• Holz David S
• Sills Maxwell

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0483

  Interaction with page-structured environments, e.g. book metaphor · CPC title

• G06F3/0485

  Scrolling or panning · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title