Wide baseline stereo for low-latency rendering

Patent metadata

Field	Value
Publication number	US-10313661-B2
Application number	US-201715459920-A
Country	US
Kind code	B2
Filing date	Mar 15, 2017
Priority date	Mar 15, 2016
Publication date	Jun 4, 2019
Grant date	Jun 4, 2019

Abstract

A virtual image generation system and method of operating same are provided. A left synthetic image and a right synthetic image of a three-dimensional scene are rendered respectively from a first left focal center and a first right focal center relative to a first viewpoint. The first left and first right focal centers are spaced from each other a distance greater than the inter-ocular distance of an end user. The synthetic image and the right synthetic image are warped respectively to a second left focal center and a second right focal center relative to a second viewpoint different from the first viewpoint. The second left and right focal centers are spaced from each other a distance equal to the inter-ocular distance of the end user. A frame is constructed from the left and right warped synthetic images, and displayed to the end user.

First claim

What is claimed is:

1. A method of operating a virtual image generation system, the method comprising:
rendering a left synthetic image and a right synthetic image of a three-dimensional scene respectively from a first left focal center and a first right focal center relative to a first head pose of an end user, the first left and first right focal centers being spaced from each other a distance greater than an inter-ocular distance of the end user;
warping the left synthetic image and the right synthetic image respectively to a second left focal center and a second right focal center relative to a second head pose of the end user different from the first head pose of the end user, the second left and right focal centers spaced from each other a distance equal to the inter-ocular distance of the end user;
constructing a frame from the left and right warped synthetic images;
displaying the frame to the end user.

2. The method of claim 1 , wherein the left and right synthetic images are rendered in three dimensions and warped in two dimensions.

3. The method of claim 1 , further comprising generating left depth data and right depth data respectively for the left synthetic image and right synthetic image, wherein the left synthetic image and the right synthetic image are respectively warped using parallax using the left depth data and the right depth data.

4. The method of claim 3 , wherein the left synthetic image and the right synthetic image are rendered based on a first look-at point in the three-dimensional scene, and the left synthetic image and the right synthetic image are warped based on the first look-at point in the three-dimensional scene and a second look-at point in the three-dimensional scene.

5. The method of claim 1 , further comprising detecting the inter-ocular distance of the user.

6. The method of claim 1 , further comprising predicting an estimate of the first head pose and detecting the second head pose.

7. The method of claim 1 , further comprising detecting each of the first and second head poses.

8. The method of claim 1 , wherein the three-dimensional scene includes at least a portion of a virtual object that is not visible from the second left and right focal centers relative to the first view point, and is visible from the second left and right focal centers relative to the second view point.

9. A virtual image generation system for use by an end user having an inter-ocular distance, comprising:
memory storing a three-dimensional scene;
a control subsystem configured for rendering a left synthetic image and a right synthetic image of the three-dimensional scene respectively from a first left focal center and a first right focal center relative to a first head pose of the end user, the first left and first right focal centers being spaced from each other a distance greater than the inter-ocular distance of the end user, the control subsystem further configured for warping the left synthetic image and the right synthetic image respectively to a second left focal center and a second right focal center relative to a second head pose different from the first head pose, the second left and right focal centers spaced from each other a distance equal to the inter-ocular distance of the end user, the control subsystem further configured for constructing a frame from the left and right warped synthetic images; and
a display system configured for displaying the frame to the end user.

10. The virtual image generation system of claim 9 , wherein the display system is configured for being positioned in front of the eyes of the end user.

11. The virtual image generation system of claim 9 , wherein the display system includes a projection subsystem and a partially transparent display surface, the projection subsystem configured for projecting the frame onto the partially transparent display surface, and the partially transparent display surface configured for being position in the field of view between the eyes of the end user and an ambient environment.

12. The virtual image generation system of claim 9 , further comprising a frame structure configured for being worn by the end user, the frame structure carrying the display system.

13. The virtual image generation system of claim 9 , wherein the control subsystem comprises a graphics control subsystem unit (GPU).

14. The virtual image generation system of claim 9 , wherein the left and right synthetic images are rendered in three dimensions and warped in two dimensions.

15. The virtual image generation system of claim 9 , wherein the control subsystem is further configured for generating left depth data and right depth data respectively for the left synthetic image and right synthetic image, wherein the left synthetic image and the right synthetic image are respectively warped using parallax using the left depth data and the right depth data.

16. The virtual image generation system of claim 15 , wherein the left synthetic image and the right synthetic image are rendered based on a first look-at point in the three-dimensional scene, and the left synthetic image and the right synthetic image are warped based on the first look-at point in the three-dimensional scene and a second look-at point in the three-dimensional scene.

17. The virtual image generation system of claim 9 , further comprising one or more sensors configured for detecting the inter-ocular distance of the user.

18. The virtual image generation system of claim 9 , further comprising a user orientation module configured for predicting an estimate of the first head pose and detecting the second head pose.

19. The virtual image generation system of claim 9 , further comprising a user orientation module configured for detecting each of the first and second head pose.

20. The virtual image generation system of claim 9 , wherein the three-dimensional scene includes at least a portion of a virtual object that is not visible from the second left and right focal centers relative to the first view point, and is visible from the second left and right focal centers relative to the second view point.

Assignees

• Magic Leap Inc

Inventors

• Kass Michael

Classifications

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• H04N13/366

  using viewer tracking · CPC title

• G02B2027/0134

  of stereoscopic type · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title