Perception based predictive tracking for head mounted displays

Patent metadata

Field	Value
Publication number	US-9348410-B2
Application number	US-201514702314-A
Country	US
Kind code	B2
Filing date	May 1, 2015
Priority date	May 30, 2013
Publication date	May 24, 2016
Grant date	May 24, 2016

Abstract

There is disclosed a method of and apparatus for predictive tracking for a head mounted display. The method comprises obtaining one or more three-dimensional angular velocity measurements from a sensor monitoring the head mounted display and setting a prediction interval based upon the one or more three-dimensional angular velocity measurements such that the prediction interval is substantially zero when the head mounted display is substantially stationary and the prediction interval increases up to a predetermined latency interval when the head mounted display is moving at an angular velocity of or above a predetermined threshold. The method further includes predicting a three-dimensional orientation for the head mounted display to create a predicted orientation at a time corresponding to the prediction interval, and generating a rendered image corresponding to the predicted orientation for presentation on the head mounted display.

First claim

What is claimed is:

1. A method comprising:
obtaining an angular velocity measurement from a sensor monitoring a head mounted display;
determining a prediction interval of time based on the angular velocity measurement and a smoothstep function, wherein the rate of increase of the determined prediction interval of time approaches zero as the determined prediction interval of time approaches either zero or a maximum latency interval;
predicting a three-dimensional orientation for the head mounted display using the smoothed angular velocity measurement extrapolated over the prediction interval of time to create a predicted orientation of the head mounted display after a passage of time equal to the prediction interval of time; and
generating a rendered image corresponding to the predicted orientation of the head mounted display for presentation on the head mounted display.

2. The method of claim 1 , wherein the smoothstep function is described by the function smoothstep(t)=3t 2 −2t 3 wherein t is an unsmoothed prediction interval scaled such that t=1 corresponds to the latency interval, and smoothstep(t) is the determined prediction interval.

3. The method of claim 1 further comprising smoothing the angular velocity measurement when the angular velocity measurement indicates that the head mounted display is substantially stationary by performing a regression on the angular velocity measurement over a sample period and applying results of the regression in place of the angular velocity measurement.

4. The method of claim 3 further comprising disabling the smoothing of the angular velocity measurement while the angular velocity measurement indicates that the head mounted display is moving at an angular velocity greater than substantially zero.

5. The method of claim 1 wherein the predetermined latency interval is between 15 milliseconds and 60 milliseconds.

6. The method of claim 1 wherein the predetermined threshold is 0.3 radians per second.

7. The method of claim 1 wherein the angular velocity measurement is obtained at a sample rate greater than or equal to 1000 Hz.

8. An apparatus comprising a non-transitory computer-readable storage medium storing executable computer instructions, and further comprising a hardware processor, the hardware processor configured to execute the computer instructions to cause the processor to:
obtain an angular velocity measurement from a sensor monitoring a head mounted display;
determine a prediction interval of time based on the angular velocity measurement and a smoothstep function, wherein the rate of increase of the determined prediction interval of time approaches zero as the determined prediction interval of time approaches either zero or a maximum latency interval;
predict a three-dimensional orientation for the head mounted display using the smoothed angular velocity measurement extrapolated over the prediction interval of time to create a predicted orientation of the head mounted display after a passage of time equal to the prediction interval of time; and
generate a rendered image corresponding to the predicted orientation of the head mounted display for presentation on the head mounted display.

9. The apparatus of claim 8 , wherein the smoothstep function is described by the function smoothstep(t)=3t 2 −2t 3 wherein t is an unsmoothed prediction interval scaled such that t=1 corresponds to the latency interval, and smoothstep(t) is the determined prediction interval.

10. The apparatus of claim 8 wherein the instructions, when executed by a processor, will further cause the processor to smooth the angular velocity measurement when the angular velocity measurement indicates that the head mounted display is substantially stationary by performing a regression on the angular velocity measurement over a sample period and applying results of the regression in place of the angular velocity measurement.

11. The apparatus of claim 10 wherein the instructions, when executed by a processor, will further cause the processor to disable the smoothing of the angular velocity measurement while the angular velocity measurement indicates that the head mounted display is moving at an angular velocity greater than substantially zero.

12. The apparatus of claim 8 wherein the predetermined latency interval is between 15 milliseconds and 60 milliseconds.

13. The apparatus of claim 8 wherein the predetermined threshold is 0.3 radians per second.

14. The apparatus of claim 8 wherein the angular velocity measurement is obtained at a sample rate greater than or equal to 1000 Hz.

15. The apparatus of claim 8 further comprising:
the sensor for providing the measurement of angular velocity;
the head mounted display for displaying the predicted image;
the processor; and
a memory;
wherein the processor and the memory comprise circuits and software for performing the instructions on the storage medium.

16. A non-transitory computer-readable storage medium storing executable computer instructions that, when executed by a processor, are configured to perform steps comprising:
obtaining an angular velocity measurement from a sensor monitoring a head mounted display;
determining a prediction interval of time based on the angular velocity measurement and a smoothstep function, wherein the rate of increase of the determined prediction interval of time approaches zero as the determined prediction interval of time approaches either zero or a maximum latency interval;
predicting a three-dimensional orientation for the head mounted display using the smoothed angular velocity measurement extrapolated over the prediction interval of time to create a predicted orientation of the head mounted display after a passage of time equal to the prediction interval of time; and
generating a rendered image corresponding to the predicted orientation of the head mounted display for presentation on the head mounted display.

17. The non-transitory computer-readable storage medium of claim 16 wherein the smoothstep function is described by the function smoothstep(t)=3t 2 −2t 3 wherein t is an unsmoothed prediction interval scaled such that t=1 corresponds to the latency interval, and smoothstep(t) is the determined prediction interval.

18. The non-transitory computer-readable storage medium of claim 16 wherein the instructions, when executed by a processor, are configured to perform further steps comprising smoothing the angular velocity measurement when the angular velocity measurement indicates that the head mounted display is substantially stationary by performing a regression on the angular velocity measurement over a sample period and applying results of the regression in place of the angular velocity measurement.

19. The non-transitory computer-readable storage medium of claim 18 wherein the instructions, when executed by a processor, are configured to perform further steps comprising disabling the smoothing of the angular velocity measurement while the angular velocity measurement indicates that the head mounted display is moving at an angular velocity greater than substantially zero.

20. The non-transitory computer-readable storage medium of claim 16 wherein the predetermined latency interval is between 15 milliseconds and 60 milliseconds.

Assignees

• Oculus Vr Llc

Inventors

• Lavalle Steve
• Giokaris Peter

Classifications

• G02B27/017 Primary

  Head mounted · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06T5/002

  Physics · mapped topic