Method and apparatus for compensating for movement of head-mounted display

Patent metadata

Field	Value
Publication number	US-10217271-B2
Application number	US-201615270409-A
Country	US
Kind code	B2
Filing date	Sep 20, 2016
Priority date	Sep 21, 2015
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

A method of compensating for a movement of a head-mounted display (HMD) includes sensing the movement of the HMD, predicting a movement of the HMD based on the sensed movement, projecting and rendering an image based on the predicted movement, re-sensing the movement of the HMD, re-predicting the movement of the HMD based on the re-sensed movement, determining an error value based on a comparison between the predicted movement of the HMD and the re-predicted movement of the HMD, and re-projecting and rendering the image based on the determined error value.

First claim

What is claimed is:

1. A method of compensating for a movement of a head-mounted display (HMD), the method comprising:
sensing the movement of the HMD;
predicting a movement of the HMD at a first eye view point based on the sensed movement;
projecting and rendering an image based on the predicted movement of the HMD at the first eye view point;
re-sensing the movement of the HMD;
re-predicting the movement of the HMD at a second eye view point based on the re-sensed movement;
determining an error value based on a comparison between the predicted movement of the HMD at the first eye view point and the re-predicted movement of the HMD at the second eye view point; and
re-projecting and rendering the image based on the determined error value,
wherein the determined error value comprises at least one of a horizontal error value, a vertical error value, and a depth error value,
wherein the re-projecting and rendering comprises:
determining a sequence of a horizontal direction re-projection based on the horizontal error value, a vertical direction re-projection based on the vertical error value, and a depth direction re-projection based on the depth error value; and
re-projecting and rendering the image in the determined sequence, and
wherein a result of projecting three dimensional (3D) geometry information on a two dimensional (2D) plane for rendering of the first eye view point is re-projected as a re-projection result on a 2D plane of the second eye view point.

2. The method of claim 1 , further comprising performing image space transformation on the re-projected and rendered image.

3. The method of claim 1 , wherein the determined error value comprises the horizontal error value, and
wherein the re-projecting and rendering comprises re-projecting and rendering the image in a horizontal direction based on the determined error value.

4. The method of claim 1 , wherein the determined error value comprises the vertical error value, and
wherein the re-projecting and rendering comprises re-projecting and rendering the image in a vertical direction based on the determined error value.

5. The method of claim 1 , wherein the determined error value comprises the depth error value, and
wherein the re-projecting and rendering comprises re-projecting and rendering the image in a depth direction based on the determined error value.

6. The method of claim 1 , wherein the re-projecting and rendering comprises re-projecting and rendering the image with respect to one view point or two view points.

7. The method of claim 1 , wherein the predicting comprises predicting the movement of the HMD at a one-eye view point or at both-eye view points, and
wherein the rendering of the image comprises rendering the image at the one-eye view point, at a view point between the both-eye view points, or at an intermediate value view point of the both-eye view points.

8. The method of claim 1 , wherein the rendering of the image comprises rasterizing and rendering the image or ray tracing and rendering the image.

9. The method of claim 1 , wherein the determining of the sequence comprises:
in response to the horizontal error value being greater than the vertical error value, determining the sequence such that the image is re-projected in a horizontal direction and then in a vertical direction; and
in response to the horizontal error value being less than the vertical error value, determining the sequence such that the image is re-projected in the vertical direction and then in the horizontal direction.

10. The method of claim 1 , wherein the determining of the sequence comprises determining the sequence based on whether at least one of the horizontal error value and the vertical error value exceeds a threshold value.

11. The method of claim 1 , wherein the determining of the sequence comprises determining the sequence for each image region.

12. The method of claim 1 , further comprising outputting the rendered image on the HMD.

13. An apparatus for compensating for a movement of a head-mounted display (HMD), the apparatus comprising:
a sensor configured to sense the movement of the HMD; and
a processor configured to predict a movement of the HMD at a first eye view point based on the movement sensed by the sensor, and project and render an image based on the predicted movement of the HMD at the first eye view point,
wherein the sensor is further configured to re-sense the movement of the HMD,
wherein the processor is further configured to re-predict the movement of the HMD at a second eye view point based on the re-sensed movement, determine an error value by comparing the predicted movement of the HMD at the first eye view point and the re-predicted movement of the HMD at the second eye view point, and re-project and render the image based on the determined error value,
wherein the determined error value comprises at least one of a horizontal error value, a vertical error value, and a depth error value,
wherein the processor is further configured to determine a sequence of a horizontal direction re-projection based on the horizontal error value, a vertical direction re-projection based on the vertical error value, and a depth direction re-projection based on the depth error value, and re-project and render the image in the determined sequence,
wherein a result of projecting three dimensional (3D) geometry information on a two dimensional (2D) plane for rendering of the first eye view point is re-projected as a re-projection result on a 2D plane of the second eye view point.

14. The apparatus of claim 13 , wherein the processor is further configured to perform image space transformation on the re-projected and rendered image.

15. The apparatus of claim 13 , wherein the determined error value comprises the horizontal error value, and wherein the processor is further configured to re-project and render the image in a horizontal direction based on the determined error value.

16. The apparatus of claim 13 , wherein the determined error value comprises the vertical error value, and
wherein the processor is further configured to re-project and render the image in a vertical direction based on the determined error value.

17. The apparatus of claim 13 , wherein the determined error value comprises the depth error value, and
wherein the processor is further configured to re-project and render the image in a depth direction based on the determined error value.

18. The apparatus of claim 13 , wherein the processor is further configured to re-project and render the image with respect to one view point or two view points.

19. The apparatus of claim 13 , wherein the processor is further configured to predict the movement of the HMD at a one-eye view point or at both-eye view points, and render the image at the one-eye view point, at a view point between the both-eye view points, or at an intermediate value view point of the both-eye view points.

20. The apparatus of claim 13 , wherein the processor is further configured to rasterize and render the image or to ray trace and render the image.

21. The apparatus of claim 13 , wherein the processor is further configured to, in response to the horizontal error value being greater than the vertical error value, determine the sequence such that the image is re-projected in a horizontal direction and then in a vertical direction, and, in response to the horizontal error value being smaller than the vertical error value, determine the sequence such that the image is re-

Assignees

• Samsung Electronics Co Ltd
• Samusung Electronics Co Ltd

Inventors

• Hwang Seokjoong
• Shin Youngsam
• Lee Wonjong

Classifications

• G09G3/003

  to produce spatial visual effects · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G02B27/017

  Head mounted · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06T15/06

  Ray-tracing · CPC title