Methods and systems for user interaction within virtual reality scene using head mounted display

Patent metadata

Field	Value
Publication number	US-10073516-B2
Application number	US-201514745267-A
Country	US
Kind code	B2
Filing date	Jun 19, 2015
Priority date	Dec 29, 2014
Publication date	Sep 11, 2018
Grant date	Sep 11, 2018

Abstract

Inertial sensors within a head mounted display are used to track movement of the head mounted display. The tracked movement of the head mounted display is correlated to an action within a virtual reality scene that is currently displayed to a user wearing the head mounted display. The action within the virtual reality scene is based on a context of the virtual reality scene that is currently displayed. The detected movement of the head mounted display can be combined with other sensor data, such as gaze detection data, to determine the action within the virtual reality scene. In this manner, movements of the user as detected using the inertial sensors within the head mounted display are used as inputs to cause actions within the current context of the virtual reality scene as displayed within the head mounted display to the user.

First claim

What is claimed is:

1. A method, comprising:
transmitting video data to a head mounted display for display of a virtual reality scene within the head mounted display;
receiving inertial sensor data from one or more inertial sensors within the head mounted display, the inertial sensor data indicating movement of the head mounted display in accordance with movement of a user by whom the head mounted display is worn, wherein the inertial sensor data provides motion information regarding a route of movement and a rate of movement of the head mounted display from a starting position when the virtual reality scene is displayed by the head mounted display;
determining if the motion information is correlated to a movement of an avatar of the user in a given direction within a current context of the virtual reality scene; and
upon determining that the motion information is correlated to the location movement of the avatar of the user in the given direction, generating an update to the virtual reality scene to reflect an ongoing movement of the avatar of the user in the given direction within the virtual reality scene;
receiving additional inertial sensor data from the one or more inertial sensors within the head mounted display, the additional inertial sensor data providing additional motion information indicating movement of the head mounted display back toward the starting position to cause stoppage of the ongoing movement of the avatar of the user in the given direction within the virtual reality scene.

2. The method as recited in claim 1 , further comprising:
querying a calibration table to identify a particular location movement within the virtual reality scene that corresponds to the information regarding the route of movement and the rate of movement of the head mounted display from the starting position, the calibration table defining associations between a number of routes of movement of the head mounted display from the starting position and resulting actions within the virtual reality scene.

3. The method as recited in claim 2 , further comprising:
correlating the rate of movement of the head mounted display from the starting position to a rate of the movement of the avatar of the user in the given direction within the virtual reality scene.

4. The method as recited in claim 1 , further comprising:
transmitting a feedback signal to the head mounted display, the feedback signal notifying the user that movement of the head mounted display from the starting position does not correspond to an allowable movement of the avatar of the user within the virtual reality scene.

5. The method as recited in claim 1 , further comprising:
receiving external inertial sensor data from one or more inertial sensors disposed on the user at locations external to the head mounted display;
using the external inertial sensor data to facilitate correlating the inertial sensor data received from the one or more inertial sensors within the head mounted display to the movement of the avatar of the user in the given direction within the virtual reality scene.

6. The method as recited in claim 1 , further comprising:
determining if the motion information is correlated to a zoom-in or a zoom-out of a viewpoint of the user within the current context of the virtual reality scene; and
upon determining that the motion information is correlated to the zoom-in or the zoom-out of the viewpoint of the user within the current context of the virtual reality scene, generating an update to the virtual reality scene to reflect implementation of the zoom-in or the zoom-out of the viewpoint of the user.

7. The method as recited in claim 1 , further comprising:
receiving gaze direction data from one or more gaze detectors within the head mounted display, the gaze direction data indicating a vision focus direction of the user upon the virtual reality scene displayed within the head mounted display; and
synchronizing the gaze direction data with the inertial sensor data to determine a synchronized vision focus direction.

8. The method as recited in claim 7 , further comprising:
using both the route of movement of the head mounted display from the starting position and the synchronized vision focus direction to determine an action within the virtual reality scene; and
generating an update to the virtual reality scene to reflect implementation of the action.

9. The method as recited in claim 1 , wherein an entirety of the virtual reality scene displayed within the head mounted display is computer generated.

10. The method as recited in claim 1 , further comprising:
determining if the motion information is correlated to a movement of an object displayed within the current context of the virtual reality scene; and
upon determining that the motion information is correlated to the movement of the object displayed within the current context of the virtual reality scene, generating an update to the virtual reality scene to reflect implementation of the movement of the object.

11. A method, comprising:
receiving video data at a head mounted display, the video data defining a virtual reality scene;
displaying the virtual reality scene within the head mounted display;
operating one or more inertial sensors within the head mounted display to generate inertial sensor data indicating movement of the head mounted display from a starting position in accordance with movement of a user by whom the head mounted display is worn;
transmitting the inertial sensor data to a computing system from which the video data is received, wherein the inertial sensor data provides for determination by the computing system of a route of movement of the head mounted display from the starting position and determination by the computing system of a rate of movement of the head mounted display from the starting position, wherein the route of movement of the head mounted display from the starting position and the rate of movement of the head mounted display from the starting position are mapped by the computing system to a location movement of an avatar of the user in a given direction within a current context of the virtual reality scene;
receiving, from the computing system, video data defining an updated virtual reality scene reflective of an ongoing movement of the avatar of the user in the given direction within the virtual reality scene;
displaying the updated virtual reality scene within the head mounted display;
operating the one or more inertial sensors within the head mounted display to generate additional inertial sensor data indicating movement of the head mounted display back toward the starting position;
transmitting the additional inertial sensor data to the computing system, wherein the additional inertial sensor data provides for determination by the computing system that the ongoing movement of the avatar of the user in the given direction within the virtual reality scene is to stop;
receiving, from the computing system, video data defining a further updated virtual reality scene reflective of stoppage of the ongoing movement of the avatar of the user in the given direction within the virtual reality scene; and
displaying the further updated virtual reality scene within the head mounted display.

12. The method as recited in claim 11 , further comprising:
receiving, from the computing system, a feedback signal indicating that the route of movement of the head mounted display from the starting position does not correspond to an allowable location movement of the avatar of the user within the current context of the virtual reality scene; and
operating the head mounted display to provide one or more of an audible

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Larsen Eric
• Umminger Frederick
• Ye Xiaoyong
• Rimon Noam
• Stafford Jeffrey Roger
• Lou Xia

Classifications

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title