System and method for near-eye light field rendering for wide field of view interactive three-dimensional computer graphics

Patent metadata

Field	Value
Publication number	US-10969740-B2
Application number	US-201815946576-A
Country	US
Kind code	B2
Filing date	Apr 5, 2018
Priority date	Jun 27, 2017
Publication date	Apr 6, 2021
Grant date	Apr 6, 2021

Abstract

A method for rendering a light field comprises projecting rays from a viewpoint positioned at a first side of a spatial light modulator (SLM) to a clipping plane positioned at an opposing side of the SLM to form an elemental view frustum within a three-dimensional scene and rendering objects within the elemental view frustum to generate components of a first elemental image for the first elemental region. The SLM may include a tiled array of non-overlapping elemental regions and a top edge and a bottom edge of a first elemental region of the non-overlapping elemental regions are intersected by the rays to form the elemental view frustum. Furthermore, the light field may include the first elemental image and additional elemental images corresponding to the array of elemental regions and each one of the additional elemental images is rendered using an additional elemental view frustum.

First claim

What is claimed is:

1. A method for rendering a light field, comprising:
projecting rays from a viewpoint positioned at a first side of a spatial light modulator (SLM) to a clipping plane positioned at an opposing side of the SLM to form an elemental view frustum within a three-dimensional scene, wherein the SLM is tiled with an array of non-overlapping elemental regions and a top edge and a bottom edge of a first elemental region of the non-overlapping elemental regions are intersected by the rays to form the elemental view frustum; and
rendering objects within the elemental view frustum to generate components of a first elemental image for the first elemental region, wherein the light field includes the first elemental image and additional elemental images corresponding to the array of elemental regions and each one of the additional elemental images is rendered using an additional elemental view frustum,
wherein the rendering comprises, for each pixel of the SLM within the first elemental region:
projecting second rays from the pixel of the SLM to the clipping plane to define a pixel diffraction cone having a base of a first width; and
removing a portion of the components of the first elemental image that are outside of the pixel diffraction cone.

2. The method of claim 1 , wherein the components include color and position in three-dimensional space.

3. The method of claim 1 , wherein the components include phase and amplitude.

4. The method of claim 3 , further comprising computing the phase and amplitude as a product of an object wave and a conjugate reference wave corresponding to a plane wave illumination source.

5. The method of claim 3 , further comprising computing the phase and amplitude as a product of an object wave and a conjugate reference wave corresponding to a spherical wave illumination source.

6. A method for rendering a light field, comprising:
computing a lateral offset between a view position and a spatial light modulator (SLM) based on a size of the SLM and a width of a holographic element, wherein an array of holographic elements covers a surface of the SLM; and
rendering object within a three-dimensional scene from the view position to produce components for each elemental image in an array of elemental images, wherein the rendering of a first elemental image of the array of elemental images comprises, for each pixel of the SLM:
projecting rays from the pixel to a clipping plane to define a pixel diffraction cone having a base, and
removing a portion of the components of the first elemental image that are outside of the pixel diffraction cone.

7. The method of claim 6 , wherein for at least one elemental image of the array of elemental images, the rendering further comprises: computing a color array and a depth array corresponding to the at least one elemental image.

8. The method of claim 7 , further comprising: calculating a phase value for a pixel of the SLM based on at least a depth value from the depth array.

9. The method of claim 7 , further comprising: calculating an amplitude value for a pixel of the SLM based on at least a corresponding color value from the color array.

10. The method of claim 6 , further comprising computing a phase and an amplitude for a pixel of the SLM as a product of an object wave and a conjugate reference wave corresponding to a spherical wave illumination source.

11. A system for rendering a light field, comprising:
a spatial light modulator (SLM); and
a processor coupled to the SLM and configured to:
project rays from a viewpoint positioned at a first side of the SLM to a clipping plane positioned at an opposing side of the SLM to form an elemental view frustum within a three-dimensional scene, wherein the SLM is tiled with an array of non-overlapping elemental regions and a top edge and a bottom edge of a first elemental region of the non-overlapping elemental regions are intersected by the rays to form the elemental view frustum; and
render objects within the elemental view frustum to generate components of a first elemental image for the first elemental region, wherein the light field includes the first elemental image and additional elemental images corresponding to the array of elemental regions and each one of the additional elemental images is rendered using an additional elemental view frustum,

wherein the rendering comprises, for each pixel of the SLM within the first elemental region:
projecting second rays from the pixel of the SLM to the clipping plane to define a pixel diffraction cone having a base of a first width; and
removing a portion of the components of the first elemental image that are outside of the pixel diffraction cone.

12. The system of claim 11 , wherein the components include color and position in three-dimensional space.

13. The system of claim 11 , wherein the components include phase and amplitude.

14. The system of claim 13 , further comprising computing the phase and amplitude as a product of an object wave and a conjugate reference wave corresponding to a plane wave illumination source.

15. The system of claim 13 , further comprising computing the phase and amplitude as a product of an object wave and a conjugate reference wave corresponding to a spherical wave illumination source.

16. The system of claim 11 , wherein the processing unit comprises a graphics processing unit.

Assignees

• Nvidia Corp

Inventors

• Shi Liang
• Huang Fu-Chung
• Lopes Ward

Classifications

• G03H1/0808 Primary

  Methods of numerical synthesis, e.g. coherent ray tracing [CRT], diffraction specific · CPC title

• G03H2210/36

  Occluded features resolved due to parallax selectivity · CPC title

• G03H2210/45

  Representation of the decomposed object · CPC title

• G03H1/2294

  Addressing the hologram to an active spatial light modulator · CPC title

• G02B27/0172

  characterised by optical features · CPC title