Processing of gesture-based user interaction using volumetric zones

Patent metadata

Field	Value
Publication number	US-10990189-B2
Application number	US-201916725406-A
Country	US
Kind code	B2
Filing date	Dec 23, 2019
Priority date	Sep 14, 2007
Publication date	Apr 27, 2021
Grant date	Apr 27, 2021

Abstract

Systems and methods for processing gesture-based user interactions within an interactive display area are provided. The display of one or more virtual objects and user interactions with the one or more virtual objects may be further provided. Multiple interactive areas may be created by partitioning an area proximate a display into multiple volumetric spaces or zones. The zones may be associated with respective user interaction capabilities. A representation of a user on the display may change as the ability of the user to interact with one or more virtual object changes.

First claim

What is claimed is:

1. A method comprising:
tracking, via a computer vision system, at least part of a hand of a user over time;
rendering, based on the tracking, a visual representation of the at least part of the hand on a display;
rendering a virtual object along with the visual representation of the at least part of the hand on the display;
detecting a distance between the at least part of the hand and at least part of a body of the user;
determining that the distance exceeds a certain threshold;
in response to determining that the distance exceeds the certain threshold, changing the rendering of the visual representation of the at least part of the hand by selecting, from an animation sequence, a predefined animation frame of the at least part of the hand that corresponds to the distance; and
detecting a selection of the virtual object based at least in part on movement of the at least part of the hand.

2. The method of claim 1 , wherein detecting the selection of the virtual object comprises detecting the selection of the virtual object based at least in part on a distance value corresponding to the distance detected between the at least part of the hand and the at least part of the body of the user.

3. The method of claim 2 , wherein the at least part of the body of the user comprises at least one of:
a head of the user;
a torso of the user;
a shoulder of the user; or
a center of gravity of the user.

4. The method of claim 1 , wherein detecting the selection of the virtual object comprises detecting the selection of the virtual object based at least in part on a distance of the hand of the user from the virtual object.

5. The method of claim 1 , wherein the animation sequence is configured to cause the visual representation to depict that the at least part of the hand gradually opens or closes as the distance between the at least part of the hand and the at least part of the body of the user increases or decreases.

6. The method of claim 1 , wherein the predefined animation frame depicts one or more fingers of the hand in a partially closed position.

7. The method of claim 1 , wherein changing the rendering of the visual representation of the at least part of the hand comprises changing the rendering of the visual representation based at least in part on an amount by which the distance between the at least part of the hand and the at least part of the body of the user a increases or decreases within a specified period of time.

8. The method of claim 1 , wherein detecting the selection of the virtual object comprises detecting the selection of the virtual object based at least in part on a speed of the hand relative to the display along an axis perpendicular to the display.

9. A system comprising:
a physical processor;
a memory device;
a display device; and
a computer vision system that has at least one camera and that is coupled to the display device, wherein the memory device stores instructions that, when executed by the physical processor, cause the system to:
track at least part of a hand of a user over time;
render, based on the tracking, a visual representation of the at least part of the hand on the display device;
render a virtual object along with the visual representation of the at least part of the hand on the display;
detect a distance between the at least part of the hand and at least part of a body of the user;
determine that the distance exceeds a certain threshold;
change, in response to determining that the distance exceeds the certain threshold, the rendering of the visual representation of the at least part of the hand by selecting, from an animation sequence, a predefined animation frame of the at least part of the hand that corresponds to the distance; and
detect a selection of the virtual object based at least in part on movement of the at least part of the hand.

10. The system of claim 9 , wherein the instructions cause the system to detect the selection of the virtual object based at least in part on a distance value corresponding to the distance detected between the at least part of the hand and the at least part of the body of the user.

11. The system of claim 10 , wherein the at least part of the body of the user comprises at least one of:
a head of the user;
a torso of the user;
a shoulder of the user; or
a center of gravity of the user.

12. The system of claim 9 , wherein the instructions cause the system to detect the selection of the virtual object based at least in part on a distance of the hand of the user from the virtual object.

13. The system of claim 9 , wherein the animation sequence is configured to cause the visual representation to depict that the at least part of the hand gradually opens or closes as the distance between the at least part of the hand and the at least part of the body of the user increases or decreases.

14. The system of claim 9 , wherein the predefined animation frame depicts one or more fingers of the hand in a partially closed position.

15. The system of claim 9 , wherein the instructions cause the system to change the rendering of the visual representation of the at least part of the hand based at least in part on an amount by which the distance between the at least part of the hand and the at least part of the body of the user increases or decreases within a specified period of time.

16. The system of claim 9 , wherein the instructions cause the system to detect the selection of the virtual object based at least in part on a speed of the hand relative to the display along an axis perpendicular to the display.

17. A non-transitory computer-readable medium having instructions recorded thereon that, when enacted by a computer processor, cause the computer processor to:
track at least part of a hand of a user over time;
render, based on the tracking, a visual representation of the at least part of the hand on a display;
render a virtual object along with the visual representation of the at least part of the hand on the display;
detect a distance between the at least part of the hand and at least part of a body of the user;
determine that the distance exceeds a certain threshold;
change, in response to determining that the distance exceeds the certain threshold, the rendering of the visual representation of the at least part of the hand by selecting, from an animation sequence, a predefined animation frame of the at least part of the hand that corresponds to the distance; and
detect a selection of the virtual object based at least in part on movement of the at least part of the hand.

18. The non-transitory computer-readable medium of claim 17 , wherein the instructions cause the computer processor to detect the selection of the virtual object based at least in part on a distance value corresponding to the distance detected between the at least part of the hand and the at least part of the body of the user.

19. The non-transitory computer-readable medium of claim 17 , wherein the instructions cause the computer processor to detect the selection of the virtual object based at least in part on a distance of the hand of the user from the virtual object.

20. The non-transitory computer-readable medium of claim 17 , wherein the instructions cause the computer processor to change the rendering of the visual representation of the at least part of the hand based at least in part on an amount by which the distance between the at least part of the hand and the at least part of the body of the user increases or decreases w

Assignees

• Facebook Inc

Inventors

• Bell Matthew
• Chennavasin Tipatat
• Clanton Charles H
• Hulme Michael
• Ophir Eyal
• Vieta Matthew

Classifications

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G01B11/14

  for measuring distance or clearance between spaced objects or spaced apertures (G01B11/26 takes precedence; rangefinders G01C3/00) · CPC title

• A63F2300/1087

  comprising photodetecting means, e.g. a camera · CPC title