Ground plane adjustment in a virtual reality environment

Patent metadata

Field	Value
Publication number	US-9766460-B2
Application number	US-201514611447-A
Country	US
Kind code	B2
Filing date	Feb 2, 2015
Priority date	Jul 25, 2014
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

An HMD device is configured to vertically adjust the ground plane of a rendered virtual reality environment that has varying elevations to match the flat real world floor so that the device user can move around to navigate and explore the environment and always be properly located on the virtual ground and not be above it or underneath it. Rather than continuously adjust the virtual reality ground plane, which can introduce cognitive dissonance discomfort to the user, when the user is not engaged in some form of locomotion (e.g., walking), the HMD device establishes a threshold radius around the user within which virtual ground plane adjustment is not performed. The user can make movements within the threshold radius without the HMD device shifting the virtual terrain. When the user moves past the threshold radius, the device will perform an adjustment as needed to match the ground plane of the virtual reality environment to the real world floor.

First claim

What is claimed:

1. A method performed by a head mounted display (HMD) device supporting rendering of a virtual reality environment, comprising:
generating sensor data describing a physical space adjoining a user of the HMD device;
using the sensor data, reconstructing a geometry of the physical space including a real world floor;
using the reconstructed geometry, determining a location of the user's head in the physical space including a height of the user's head from the real world floor;
establishing a threshold radius around the location;
maintaining a ground plane for the virtual reality environment when the location is within the threshold radius; and
adjusting the ground plane to match the real world floor when the location is outside the threshold radius.

2. The method of claim 1 in which the sensor data includes depth data and further including generating the sensor data using a depth sensor and applying surface reconstruction techniques to reconstruct the physical space geometry.

3. The method of claim 1 further including generating depth data using one or more depth-from-stereo imaging analyses.

4. The method of claim 1 further including varying the threshold radius according to terrain type in the virtual reality environment.

5. The method of claim 1 further including configuring the HMD device to expose a user interface (UI) to the user and performing the adjusting responsively to user input to the UI.

6. The method of claim 1 further including performing the adjusting depending on a traversal path of the user in the virtual reality environment, the path being one of a path having increasing virtual elevation or a path having decreasing virtual elevation.

7. The method of claim 6 further including performing the adjusting for each of one or more discrete chunks of the virtual reality environment along a traversal path.

8. The method of claim 1 further including applying smoothing or filtering when adjusting ground plane.

9. The method of claim 1 further including determining the height using surface reconstruction data obtained when the user looks down at the virtual reality environment ground plane.

10. The method of claim 1 in which the threshold radius has a value between one-quarter and three-quarters of a meter.

11. A head mounted display (HMD) device operable by a user in a physical environment having a real world floor, comprising:
one or more processors;
a display for rendering a virtual reality environment to the user; and
one or more memory devices storing computer-readable instructions which, when executed by the one or more processors, perform a method comprising the steps of:
generating surface reconstruction data using a depth sensor or camera system,
using the surface reconstruction data, determining a height of the user's head from the real world floor,
using the surface reconstruction data, detecting when the user has moved beyond a threshold radius that is established around the user,
operating the HMD device to render the virtual reality environment on the display so that the ground plane is aligned with the real world floor based on the determined height when the user has moved beyond the threshold radius.

12. The HMD device of claim 11 further including using the surface reconstruction data to determine whether the user is traversing the virtual reality environment and establishing the threshold radius when the user is not traversing the virtual reality environment.

13. The HMD device of claim 12 further including performing the operating responsively to user input, virtual terrain type, and direction of traversal in the virtual reality environment.

14. The HMD device of claim 11 further including varying the threshold radius when the user is proximate to a portion of the virtual reality environment that includes abrupt changes in elevation.

15. The HMD device of claim 11 further including modeling the physical environment using a surface reconstruction data pipeline that implements a volumetric method creating multiple overlapping surfaces that are integrated.

16. The HMD device of claim 11 further including a sealed protective visor that substantially encases the display.

17. One or more hardware-based computer-readable memory devices not consisting of propagated signals storing computer-executable instructions for rendering a virtual reality environment having variable virtual elevations on a head mounted display (HMD) device located in a physical space having a floor, the method comprising the steps of:
using a depth sensor to generate surface reconstruction data to dynamically generate and update a model of the physical space as a user traverses the virtual reality environment;
determining a height of the user's head from the floor of the physical space using the model; and
using the determined height, vertically adjusting a ground plane of the virtual reality environment to align with the floor of the physical space.

18. The one or more hardware-based computer-readable memory devices of claim 17 further including performing the vertical adjustment on a frame-by-frame basis or a group of frames basis.

19. The one or more hardware-based computer-readable memory devices of claim 18 further including filtering or smoothing the vertical adjustment.

20. The one or more hardware-based computer-readable memory devices of claim 19 further including using the model to determine a location of the user's head in the physical space, establishing a threshold radius around a location, maintaining a current alignment of the ground plane when the location is within the threshold radius, and performing the vertical adjustment of the ground plane when the location is beyond the threshold radius.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Burns Aaron
• Sugden Ben
• Massey Laura
• Da Veiga Alexandre
• Salter Tom
• Alt Greg

Classifications

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• G06T7/73

  using feature-based methods · CPC title

• H04N13/279

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B2027/0167

  Emergency system, e.g. to prevent injuries · CPC title