Methods and Systems for User Interaction within Virtual Reality Scene using Head Mounted Display

Patent metadata

Field	Value
Publication number	US-2016187969-A1
Application number	US-201514745267-A
Country	US
Kind code	A1
Filing date	Jun 19, 2015
Priority date	Dec 29, 2014
Publication date	Jun 30, 2016
Grant date	—

Abstract

Inertial sensors within a head mounted display are used to track movement of the head mounted display. The tracked movement of the head mounted display is correlated to an action within a virtual reality scene that is currently displayed to a user wearing the head mounted display. The action within the virtual reality scene is based on a context of the virtual reality scene that is currently displayed. The detected movement of the head mounted display can be combined with other sensor data, such as gaze detection data, to determine the action within the virtual reality scene. In this manner, movements of the user as detected using the inertial sensors within the head mounted display are used as inputs to cause actions within the current context of the virtual reality scene as displayed to the user within the head mounted display.

First claim

What is claimed is:

1 . A method, comprising:
transmitting video data to a head mounted display for display of a virtual reality scene within the head mounted display; receiving inertial sensor data from one or more inertial sensors within the head mounted display, the inertial sensor data indicating movement of the head mounted display in accordance with movement of a user by whom the head mounted display is worn, the movement of the user based on the virtual reality scene displayed within the head mounted display; correlating the received inertial sensor data to an action within the virtual reality scene, wherein the correlating includes determining a route of movement of the head mounted display and determining a rate of movement of the head mounted display, wherein the correlating includes determining a set of allowable virtual movements within a current context of the virtual reality scene, and wherein the correlating includes mapping the route of movement of the head mounted display to a corresponding virtual movement within the set of allowable virtual movements; and generating an update to the virtual reality scene to reflect implementation of the action.

2 . The method as recited in claim 1 , wherein mapping the route of movement of the head mounted display to the corresponding virtual movement includes querying a calibration table, the calibration table defining associations between a number of routes of movement of the head mounted display and resulting actions within the virtual reality scene.

3 . The method as recited in claim 2 , wherein the correlating includes mapping the rate of movement of the head mounted display to a rate of the corresponding virtual movement within the virtual reality scene.

4 . The method as recited in claim 1 , further comprising:
transmitting a feedback signal to the head mounted display, the feedback signal notifying the user that the movement of the head mounted display does not correspond to any virtual movement within the set of allowable virtual movements as determined by the correlating.

5 . The method as recited in claim 1 , further comprising:
receiving external inertial sensor data from one or more inertial sensors disposed on the user at locations external to the head mounted display; using the external inertial sensor data to facilitate correlating the inertial sensor data received from the one or more inertial sensors within the head mounted display to the action within the virtual reality scene.

6 . The method as recited in claim 1 , wherein the action within the virtual reality scene is one or more of a movement of a viewpoint of the user within the virtual reality scene, a zoom-in or a zoom-out of a viewpoint of the user within the virtual reality scene, and a movement of an object displayed within the virtual reality scene.

7 . The method as recited in claim 1 , further comprising:
receiving gaze direction data from one or more gaze detectors within the head mounted display, the gaze direction data indicating a vision focus direction of the user upon the virtual reality scene displayed within the head mounted display; and synchronizing the gaze direction data with the inertial sensor data to determine a synchronized vision focus direction.

8 . The method as recited in claim 7 , wherein the correlating includes determining a route of movement of the head mounted display and determining a rate of movement of the head mounted display, and wherein the correlating includes using both the route of movement of the head mounted display and the synchronized vision focus direction to determine the action within the virtual reality scene.

9 . A method, comprising:
receiving video data at a head mounted display, the video data defining a virtual reality scene; displaying the virtual reality scene within the head mounted display; operating one or more inertial sensors within the head mounted display to generate inertial sensor data indicating movement of the head mounted display in accordance with movement of a user by whom the head mounted display is worn, the movement of the user corresponding to an action within the virtual reality scene; transmitting the inertial sensor data to a computing system from which the video data is received, wherein the inertial sensor data provides for determination of a route of movement of the head mounted display by the computing system and determination of a rate of movement of the head mounted display by the computing system, wherein the route of movement of the head mounted display and the rate of movement of the head mounted display are mapped by the computing system to a corresponding virtual movement within a set of allowable virtual movements within a current context of the virtual reality scene, the corresponding virtual movement directing the action within the virtual reality scene; receiving, from the computing system, video data defining an updated virtual reality scene reflective of the action within the virtual reality scene corresponding to the movement of the user; and displaying the updated virtual reality scene within the head mounted display.

10 . The method as recited in claim 9 , further comprising:
receiving, from the computing system, a feedback signal indicating that the route of movement of the head mounted display does not have any corresponding virtual movement within the set of allowable virtual movements within the current context of the virtual reality scene; and operating the head mounted display to provide one or more of an audible signal, a tactile signal, and a visual signal to notify the user of receipt of the feedback signal.

11 . The method as recited in claim 9 , further comprising:
transmitting external inertial sensor data to the computing system from one or more inertial sensors disposed on the user at locations external to the head mounted display, the external inertial sensor data used by the computing system to facilitate determination of the route of movement of the head mounted display.

12 . The method as recited in claim 9 , wherein the action within the virtual reality scene is one or more of a movement of a viewpoint of the user within the virtual reality scene, a zoom-in or a zoom-out of a viewpoint of the user within the virtual reality scene, and a movement of an object displayed within the virtual reality scene.

13 . The method as recited in claim 9 , further comprising:
operating one or more gaze detectors within the head mounted display to generate gaze direction data indicating a vision focus direction of the user upon the virtual reality scene displayed within the head mounted display; and transmitting the gaze direction data to the computing system from which the video data is received, the gaze direction data synchronized with the inertial sensor data to determine a synchronized vision focus direction.

14 . The method as recited in claim 13 , wherein both the route of movement of the head mounted display and the synchronized vision focus direction are used to determine the action within the virtual reality scene corresponding to the movement of the user.

15 . A system, comprising:
an inertial data processing module configured to receive inertial sensor data from a head mounted display, the inertial sensor data indicating movement of the head mounted display in accordance with movement of a user by whom the head mounted display is worn, the movement of the user based on a virtual reality scene displayed within the head mounted display, the inertial data processing module configured to determine from the inertial sensor data a route of mo

Assignees

• Sony Comp Entertainment Us

Inventors

• Larsen Eric
• Umminger Frederick
• Ye Xiaoyong
• Rimon Noam
• Stafford Jeffrey Roger
• Lou Xia

Classifications

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G02B27/017

  Head mounted · CPC title