Light field offset rendering

Patent metadata

Field	Value
Publication number	US-12374031-B2
Application number	US-202217663943-A
Country	US
Kind code	B2
Filing date	May 18, 2022
Priority date	May 18, 2022
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

A method for light field rendering of a 3D scene data. The rendering method comprises capturing a light field image at a retracted plane, parallel to the display plane, decoding a light field camera to produce a hogel camera for each hogel at the retraction plane to produce an integral image, and applying a pixel remapping technique to the pixels of the integral image for display on a light field display. The 3D scene is captured and remapped to adjust the perspective giving the illusion the image was taken at the display plane, allowing captured images to be displayed in both the inner frustum and outer frustum of a light field display, creating an immersive experience.

First claim

We claim:

1. A computer-implemented light field rendering method comprising:
defining a display plane for a light field display with a spatial resolution and a directional resolution, the light field display comprising an inner frustum volume bounded by the display plane and a far clip plane and an outer frustum volume bounded by the display plane and a near clip plane;
defining a retraction plane parallel to and at an integral offset distance from the display plane, the retraction plane comprising a plurality of light field cameras, each camera having a focal length and spaced at a sample gap, wherein the integral offset distance is calculated from the focal length, directional resolution of the light field display, and an offset integer, N≥1;
capturing a view of the 3D scene as a source image at each of the plurality of light field cameras;
decoding each source image to generate a plurality of hogel cameras on the retraction plane, each hogel camera providing an elemental image;
generating an integral image comprising a plurality of pixels from the elemental images at the retraction plane; and
executing a pixel remapping technique on individual pixels in the integral image to create a rendered light field image at the display plane.

2. The method of claim 1 , wherein the view of the 3D scene captured at the plurality of light field cameras at the retraction plane comprises image information from the inner frustum volume and the outer frustum volume.

3. The method of claim 1 , wherein the captured view of the 3D scene comprises all of the image information in the outer frustum volume.

4. The method of claim 1 , wherein the retraction plane is positioned at the near clip plane.

5. The method of claim 1 , further comprising displaying the rendered light field image on a light field display.

6. The method of claim 1 , wherein the retraction plane surface area is greater than the display plane surface area.

7. The method of claim 1 , wherein the plurality of light field cameras have optical properties comprising orientation, lens pitch, directional resolution, and field of view.

8. The method of claim 1 , further comprising generating a plurality of integral images at a plurality of retraction planes.

9. The method of claim 8 , further comprising compositing the plurality of integral images to create a composited rendered light field image at the display plane.

10. The method of claim 9 , wherein compositing incorporates transparency data.

11. The method of claim 1 , wherein each light field camera is one of a digital single reflex mirror (DSLR) camera, pinhole camera, plenoptic camera, compact camera, and mirrorless camera.

12. The method of claim 1 , wherein each light field camera is a computer-generated camera.

13. The method of claim 1 , wherein the pixel remapping technique causes the pixels to change their hogel index (H x , H y ) from the retraction plane to the display plane.

14. The method of claim 1 , wherein the retraction plane is outside of the outer frustum volume.

15. A computer-implemented method of displaying a light field image comprising:
capturing a first light field at a retraction plane relative to a light field display plane using a light field camera, the first light field comprising an array of retraction plane hogels, each hogel having a plurality of pixels, the retraction plane positioned at an integral offset distance from the display plane, the integral offset distance calculated as an offset integer N, where N≥1;
assigning a hogel index (H x , H y ) and a pixel index (P x , P y ) to each pixel in each retraction plane hogel to indicate its position in the light field display by applying a pixel remapping technique to select a single pixel from each retraction plane hogel;
loading each pixel from the light field at the retraction plane (LF r ) and storing each pixel to the light field at the display plane (LF d ) using a compositing function; and
generating a light field image at the display plane comprising remapped pixels.

16. The method of claim 15 , wherein one pixel from the retraction plane produces one pixel at the display plane.

17. The method of claim 15 , wherein applying a pixel remapping technique changes the hogel index (H x , H y ) of each pixel in each retraction plane hogel and the pixel index (P x , P y ) remains constant.

18. The method of claim 17 , wherein the pixel remapping technique is based on the equation LF r [H x +(DR x *N)−(N*P x ), H y +(DR y *N)−(N*P y ), P x , P y ]⇒LF d [H x , H y , P x , P y ].

19. The method of claim 15 , wherein the pixel remapping technique is a function of the directional resolution of the light field display and the offset integer, N.

20. The method of claim 15 , wherein the retraction plane is comprised of a sufficient number of hogels to provide the number of pixels to achieve the required directional resolution of light field display at the display plane.

21. The method of claim 15 , wherein the size of the display plane is defined by the directional resolution and spatial resolution of the light field display.

22. The method of claim 15 , wherein the light field camera is a mirror (DSLR) camera, pinhole camera, plenoptic camera, compact camera, or mirrorless camera.

Assignees

• Avalon Holographics Inc

Inventors

• Butyn Thomas
• Smith Colton

Classifications

• G06T7/557

  from light fields, e.g. from plenoptic cameras · CPC title

• G06T15/205 Primary

  Image-based rendering · CPC title

• G06T15/506 Primary

  Illumination models · CPC title