Perception based predictive tracking for head mounted displays

Patent metadata

Field	Value
Publication number	US-9063330-B2
Application number	US-201414285470-A
Country	US
Kind code	B2
Filing date	May 22, 2014
Priority date	May 30, 2013
Publication date	Jun 23, 2015
Grant date	Jun 23, 2015

Abstract

There is disclosed a method of and apparatus for predictive tracking for a head mounted display. The method comprises obtaining one or more three-dimensional angular velocity measurements from a sensor monitoring the head mounted display and setting a prediction interval based upon the one or more three-dimensional angular velocity measurements such that the prediction interval is substantially zero when the head mounted display is substantially stationary and the prediction interval increases up to a predetermined latency interval when the head mounted display is moving at an angular velocity of or above a predetermined threshold. The method further includes predicting a three-dimensional orientation for the head mounted display to create a predicted orientation at a time corresponding to the prediction interval, and generating a rendered image corresponding to the predicted orientation for presentation on the head mounted display.

First claim

It is claimed:

1. A method of predictive tracking for a head mounted display comprising:
obtaining one or more three-dimensional angular velocity measurements from a sensor monitoring a head mounted display;
setting a prediction interval based upon the one or more three-dimensional angular velocity measurements such that the prediction interval is substantially zero when the one or more angular velocity measurements indicate that the head mounted display is substantially stationary and the prediction interval increases up to a predetermined latency interval when the one or more angular velocity measurements indicate that the head mounted display is moving at an angular velocity at or above a predetermined threshold, the prediction interval being equal to or less than the predetermined latency interval;
predicting a three-dimensional orientation for the head mounted display using the one or more three-dimensional angular velocity measurements extrapolated over the prediction interval to create a predicted orientation at a time corresponding to the prediction interval; and
generating a rendered image corresponding to the predicted orientation for presentation on the head mounted display.

2. The method of claim 1 wherein setting a prediction interval includes a fully dynamic adjustment as the one or more angular velocity measurements indicate that the head mounted display is increasing in angular velocity such that the prediction interval is linearly increased over the course of the increase in angular velocity from substantially zero to the predetermined latency interval.

3. The method of claim 1 further comprising smoothing the one or more angular velocity measurements while the one or more angular velocity measurements indicate that the head mounted display is substantially stationary by performing a regression on the angular velocity measurements over a sample period and applying results of the regression in place of the one or more angular velocity measurements.

4. The method of claim 3 further comprising disabling the smoothing of the one or more angular velocity measurements while the one or more angular velocity measurements indicate that the head mounted display is moving at an angular velocity greater than substantially zero.

5. The method of claim 1 wherein the predetermined latency interval is between 15 milliseconds and 60 milliseconds.

6. The method of claim 1 wherein the predetermined threshold is 0.3 radians per second.

7. The method of claim 1 wherein the one or more angular velocity measurements are obtained at a sample rate greater than or equal to 1000 Hz.

8. An apparatus comprising a storage medium storing a program having instructions which when executed by a processor will cause the processor to:
obtain one or more three-dimensional angular velocity measurements from a sensor monitoring a head mounted display;
set a prediction interval based upon the one or more three-dimensional angular velocity measurements such that the prediction interval is substantially zero when the one or more angular velocity measurements indicate that the head mounted display is substantially stationary and the prediction interval increases up to a predetermined latency interval when the one or more angular velocity measurements indicate that the head mounted display is moving at an angular velocity at or above a predetermined threshold, the prediction interval being equal to or less than the predetermined latency interval;
predict a three-dimensional orientation for the head mounted display using the one or more three-dimensional angular velocity measurements extrapolated over the prediction interval to create a predicted orientation at a time corresponding to the prediction interval; and
generate a rendered image corresponding to the predicted orientation for presentation on the head mounted display.

9. The apparatus of claim 8 wherein the setting a prediction interval includes a fully dynamic adjustment as the one or more angular velocity measurements indicate that the head mounted display is increasing in angular velocity such that the prediction interval is linearly increased over the course of the increase in angular velocity from substantially zero to the predetermined latency interval.

10. The apparatus of claim 8 wherein the instructions, when executed by a processor, will further cause the processor to smooth the one or more angular velocity measurements while the one or more angular velocity measurements indicate that the head mounted display is substantially stationary by performing a regression on the angular velocity measurements over a sample period and applying results of the regression in place of the one or more angular velocity measurements.

11. The apparatus of claim 10 wherein the instructions, when executed by a processor, will further cause the processor to disable the smoothing of the one or more angular velocity measurements while the one or more angular velocity measurements indicate that the head mounted display is moving at an angular velocity greater than substantially zero.

12. The apparatus of claim 8 wherein the predetermined latency interval is between 15 milliseconds and 60 milliseconds.

13. The apparatus of claim 8 wherein the predetermined threshold is 0.3 radians per second.

14. The apparatus of claim 8 wherein the one or more angular velocity measurements are obtained at a sample rate greater than or equal to 1000 Hz.

15. The apparatus of claim 8 further comprising:
the sensor for providing the one or more measurements of angular velocity;
the head mounted display for displaying the predicted image;
the processor; and
a memory;
wherein the processor and the memory comprise circuits and software for performing the instructions on the storage medium.

16. A method of predictive tracking of head movement comprising:
obtaining one or more three-dimensional angular velocity measurements from a sensor monitoring a head mounted display;
setting a prediction interval based upon the one or more three-dimensional angular velocity measurements such that the prediction interval is substantially zero when the one or more angular velocity measurements indicate that the head mounted display is substantially stationary and the prediction interval increases up to a latency interval corresponding to an average latency of the head mounted display when the one or more angular velocity measurements indicate that the head mounted display is moving at an angular velocity at or above a predetermined threshold, the prediction interval being equal to or less than the predetermined latency interval;
predicting a three-dimensional orientation for the head mounted display using the one or more three-dimensional angular velocity measurements extrapolated over the prediction interval to create a predicted orientation at a time corresponding to the prediction interval; and
generating a rendered image corresponding to the predicted orientation for presentation on the head mounted display at the next available frame update for the head mounted display.

17. A method comprising:
obtaining an angular velocity measurement from a sensor monitoring a head mounted display;
determining a prediction interval of time based upon the angular velocity measurement, wherein the determined prediction interval of time is greater than a threshold interval of time when the angular velocity measurement is greater than a predetermined threshold, the prediction interval of time being equal to or less than a predetermined latency interval of time;
predicting a three-dimensiona

Assignees

• Oculus Vr Llc

Inventors

• Lavalle Steve
• Giokaris Peter

Classifications

• G02B27/017 Primary

  Head mounted · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title