System and method for near-eye light field rendering for wide field of view interactive three-dimensional computer graphics

Patent metadata

Field	Value
Publication number	US-11747766-B2
Application number	US-202117189960-A
Country	US
Kind code	B2
Filing date	Mar 2, 2021
Priority date	Jun 27, 2017
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A method for rendering a light field comprises projecting rays from a viewpoint positioned at a first side of a spatial light modulator (SLM) to a clipping plane positioned at an opposing side of the SLM to form an elemental view frustum within a three-dimensional scene and rendering objects within the elemental view frustum to generate components of a first elemental image for the first elemental region. The SLM may include a tiled array of non-overlapping elemental regions and a top edge and a bottom edge of a first elemental region of the non-overlapping elemental regions are intersected by the rays to form the elemental view frustum. Furthermore, the light field may include the first elemental image and additional elemental images corresponding to the array of elemental regions and each one of the additional elemental images is rendered using an additional elemental view frustum.

First claim

What is claimed is:

1. A method for rendering a light field that comprises a plurality of elemental images arranged in a grid, the method comprising:
generating, by a processor, for each elemental image of the plurality of elemental images, components of the light field corresponding to that elemental image by:
projecting rays from a viewpoint positioned at a lateral offset from a first side of a spatial light modulator (SLM) to form an elemental view frustum within a three-dimensional scene; and
rendering objects within the elemental view frustum to generate components of the elemental image,
wherein each elemental image of the plurality of elemental images corresponds to a holographic element (hogel) associated with one or more picture elements (pixels) of the SLM, and wherein the SLM is divided into a plurality of hogels having size w h and the lateral offset is calculated based on the size w h .

2. The method of claim 1 , wherein the rendering comprises, for each pixel of the hogel corresponding to the elemental image:
projecting second rays from the pixel to a clipping plane to define a pixel diffraction cone having a base of a first width; and
removing a portion of the components of the elemental image outside of the pixel diffraction cone.

3. The method of claim 1 , wherein the components include a color and a position in three-dimensional (3D) space.

4. The method of claim 1 , wherein the components include a phase and an amplitude.

5. The method of claim 4 , further comprising computing the phase and the amplitude as a product of an object wave and a conjugate reference wave corresponding to a plane wave illumination source.

6. The method of claim 4 , further comprising computing the phase and the amplitude as a product of an object wave and a conjugate reference wave corresponding to a spherical wave illumination source.

7. A method for rendering a light field that comprises a plurality of elemental images arranged in a grid, the method comprising:
computing, by a processor, a lateral offset based on a size w h of a holographic element (hogel) associated with a spatial light modulator (SLM), a pixel pitch of the SLM, and a wavelength associated with a monochromatic light source, wherein the SLM is divided into a plurality of hogels, each hogel associated with one or more picture elements (pixels) of the SLM; and
for each elemental image, rendering, by the processor, a three-dimensional scene from a viewpoint located at the lateral offset from a first side of the SLM to produce components of the elemental image, wherein each elemental image is associated with a different viewpoint.

8. The method of claim 7 , wherein the rendering comprises computing a color array and a depth array for at least one elemental image of the plurality of elemental images.

9. The method of claim 8 , further comprising: calculating a phase value for a pixel of the SLM based on at least one depth value of the depth array.

10. The method of claim 8 , further comprising: calculating an amplitude value for a pixel of the SLM based on at least a corresponding color value of the color array.

11. The method of claim 7 , further comprising computing a phase and an amplitude for a pixel of the SLM as a product of an object wave and a conjugate reference wave corresponding to a spherical illumination source.

12. The method of claim 7 , wherein the rendering comprises, for each pixel of the hogel corresponding to the elemental image:
projecting rays from the pixel to a clipping plane to define a pixel diffraction cone having a base of a first width.

13. The method of claim 12 , wherein the rendering further comprises removing a portion of the components of the elemental image outside of the pixel diffraction cone.

14. A system for rendering a light field that comprises a plurality of elemental images arranged in a grid, the system comprising:
a spatial light modulator (SLM) divided into a plurality of holographic elements (hogels) having size w h , each hogel associated with one or more picture elements (pixels) of the SLM, and each elemental image of the plurality of elemental images corresponding to a particular hogel of the plurality of hogels; and
a processor coupled to the SLM and configured to:
generate, for each elemental image of the plurality of elemental images, components of the light field corresponding to that elemental image by:
projecting rays from a viewpoint positioned at a lateral offset from a first side of a spatial light modulator (SLM) to form an elemental view frustum within a three-dimensional scene, wherein the lateral offset is calculated based on the size w h ; and
rendering objects within the elemental view frustum to generate components of the elemental image.

15. The system of claim 14 , wherein the rendering comprises, for each pixel of the hogel corresponding to the elemental image:
projecting second rays from the pixel to a clipping plane to define a pixel diffraction cone having a base of a first width; and
removing a portion of the components of the elemental image outside of the pixel diffraction cone.

16. The system of claim 14 , wherein the components include a color and a position in three-dimensional (3D) space.

17. The system of claim 14 , wherein the components include a phase and an amplitude.

18. The system of claim 17 , further comprising computing the phase and the amplitude as a product of an object wave and a conjugate reference wave corresponding to a plane wave illumination source.

19. The system of claim 17 , further comprising computing the phase and the amplitude as a product of an object wave and a conjugate reference wave corresponding to a spherical wave illumination source.

20. The system of claim 14 , wherein the processor comprises a graphics processing unit.

Assignees

• Nvidia Corp

Inventors

• Huang Fu-Chung
• Lopes Ward
• Shi Liang

Classifications

• G03H1/02 Primary

  Details {of features involved during the holographic process; Replication of holograms without interference recording} · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G03H1/0808 Primary

  Methods of numerical synthesis, e.g. coherent ray tracing [CRT], diffraction specific · CPC title

• G03H1/2294

  Addressing the hologram to an active spatial light modulator · CPC title

• G02B2027/0123

  comprising devices increasing the field of view · CPC title