Method, system and device for navigating in a virtual reality environment

Patent metadata

Field	Value
Publication number	US-9972136-B2
Application number	US-201715433952-A
Country	US
Kind code	B2
Filing date	Feb 15, 2017
Priority date	Dec 18, 2014
Publication date	May 15, 2018
Grant date	May 15, 2018

Abstract

A method, a system, and a device for navigating in a virtual reality scene, using body parts gesturing and posturing are provided herein. The method may include: projecting a synthetic 3D scene, into both eyes of a user, via a near eye display, so as to provide a virtual reality view to the user; identifying at least one gesture or posture carried out by at least one body part of said user; measuring at least one metric of a vector associated with the detected gesture or posture; applying a movement or action of said user in virtual reality environment, based on the measured metrics; and modifying the virtual reality view so as to reflect the movement or action of said user in the virtual reality environment.

First claim

The invention claimed is:

1. A method comprising:
displaying a synthetic 3D scene;
identifying at least one gesture or posture carried out by at least one body part of the user;
deriving a vector that spatially represents the identified gesture or posture;
transforming the vector into a continuous movement or action in the synthetic 3D scene, based on at least one of an angle of the vector and a length of the vector, wherein a speed of the movement is based on a value of the angle; and
modifying the synthetic 3D scene to reflect the movement or action in the synthetic 3D scene.

2. The method of claim 1 , wherein the identifying the at least one gesture or posture comprises applying a classifier to a database of predefined postures and gestures.

3. The method of claim 1 , wherein the movement or action in the synthetic 3D scene is carried out continuously as long as the at least one of the angle of the vector and the length of the vector maintains a value beyond a predefined threshold.

4. The method of claim 1 , wherein the movement or action in the synthetic 3D scene is terminated responsive to detecting a predefined termination act.

5. The method of claim 1 , wherein the movement or action in the synthetic 3D scene is terminated responsive to detecting a return of the at least one body part to a basic posture position.

6. The method of claim 1 , wherein the identified posture is leaning forward a torso of the user or raising shoulders of the user.

7. The method of claim 1 , wherein the identified posture is defined by a spatial relationship between at least two body parts of the user, and the method further comprises determining angles between the at least two body parts and transforming the determined angles into the continuous movement or action in the synthetic 3D scene.

8. The method of claim 1 , wherein the vector associated with the identified gesture or posture further comprises a spatial direction angle.

9. The method of claim 1 , wherein identifying the least one gesture or posture further comprises subtracting movement components affiliated with head movements of the user to retrieve postures and gestures of the at least one body part that are relative to the head of the user.

10. The method of claim 1 , further comprising superimposing a virtual user-interface object into the synthetic 3D scene enabling the user to apply gestures and postures relative to the virtual user-interface object.

11. A system comprising:
a device configured to display a synthetic 3D scene; and
a computer processor configured to:
identify at least one gesture or posture carried out by at least one body part of the user;
derive a vector that spatially represents the identified gesture or posture;
transform the vector into a continuous movement or action in the synthetic 3D scene, based on at least one of an angle of the vector and a length of the vector, wherein a speed of the movement is based on a value of the angle; and
modify the synthetic 3D scene to reflect the movement or action in the synthetic 3D scene.

12. The system of claim 11 , wherein the computer processor is further configured to apply a classifier to a database of predefined postures and gestures to identify the at least one gesture or posture.

13. The system of claim 11 , wherein the computer processor is further configured to perform the movement or action in the synthetic 3D scene continuously as long as the at least one of the angle of the vector and the length of the vector maintains a value beyond a predefined threshold.

14. The system of claim 11 , wherein the computer processor is further configured to terminate the movement or action in the synthetic 3D scene responsive to detection of a predefined termination act.

15. The system of claim 11 , wherein the computer processor is further configured to terminate the movement or action in the synthetic 3D scene responsive to detecting a return of the at least one body part to a basic posture position.

16. The system of claim 11 , wherein the identified posture is leaning forward a torso of the user or raising shoulders of the user.

17. The system of claim 11 , wherein the identified posture is defined by a spatial relationship between at least two body parts of the user, and the computer processor is further configured to determine angles between the at least two body parts and transform the determined angles into the continuous movement or action in the synthetic 3D scene.

18. The system of claim 11 , wherein the vector associated with the identified gesture or posture further comprises a spatial direction angle.

19. The system of claim 11 , wherein the computer processor is further configured to subtract movement components affiliated with head movements of the user to retrieve postures and gestures of the at least one body part that are relative to the head of the user.

20. The system of claim 11 , wherein the computer processor is further configured to superimpose a virtual user-interface object into the synthetic 3D scene enabling the user to apply gestures and postures relative to the virtual user-interface object.

Assignees

• Facebook Inc

Inventors

• Grossinger Nadav
• Alon Emil
• Bar David Iddo
• Solodnik Efrat

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title