Methods and apparatus for full parallax light field display systems

Patent metadata

Field	Value
Publication number	US-10528004-B2
Application number	US-201916413415-A
Country	US
Kind code	B2
Filing date	May 15, 2019
Priority date	Apr 23, 2015
Publication date	Jan 7, 2020
Grant date	Jan 7, 2020

Abstract

A 3D video processing system with integrated display is described wherein the huge data bandwidth demands on the source-to-display transmission medium is decreased by utilizing innovative 3D light field video data compression at the source along with innovative reconstruction of 3D light field video content from highly compressed 3D video data at the display. The display incorporates parallel processing pipelines integrated with a Quantum Photonics Imager® for efficient data handling and light imaging.

First claim

What is claimed is:

1. A system for reproducing a light field for display from a compressed light field, the system comprising:
at least one processing node having a plurality of hardware modules configured for decompressing the compressed light field and reproducing the light field for a display;
each processing node having:
a data decoder, and
a plurality of depth-image-based rendering (DIBR) modules;

each processing node having a bus interconnect for interconnecting the modules and a plurality of memories; and
a sequence controller for controlling a sequence of operation of the modules to decompress compressed light field data to reproduce the light field for display.

2. The system of claim 1 , wherein at least one of the modules is an application specific integrated circuit.

3. The system of claim 2 wherein at least one of the modules is programmable.

4. The system of claim 1 , wherein each processing node also has an interface module configured to receive data packets and deliver the data packets to the data decoder.

5. The system of claim 4 , wherein the data decoder is configured to receive data packets from the interface module, and for each data packet, if a packet header of the data packet indicates that the data packet needs to be decoded, the data decoder is further configured to decode the data packet in accordance with a packet type.

6. The system of claim 1 , wherein each processing module also has an inverse image transform module configured to receive from the data decoder for blockwise seed and residual texture decoding and dequantization.

7. The system of claim 6 wherein the inverse image transform module uses a predefined image transform matrix, thereby allowing an inverse transform instruction to serve as a pipelined single instruction, multiple data instruction.

8. The system of claim 1 , wherein each processing module also has a hogel content repetition module configured to copy data between internal buffers having different lengths and different data widths without multiplicative scaling.

9. The system of claim 1 , wherein the plurality of DIBR modules comprises a forward DIBR module configured to receive seed hogel disparity and produce warped disparity.

10. The system of claim 9 , wherein the plurality of DIBR modules comprises a backward DIBR module configured to read a generated temporary disparity from the forward DIBR module and calculate a current hogel address reference position in a seed hogel texture and generate hogels.

11. The system of claim 10 , wherein the backward DIBR module is configured to also use residual disparity to combine with the seed hogel disparity to fix disparity errors.

12. The system of claim 1 , wherein the plurality of DIBR modules comprises a backward DIBR module configured to fill pixel positions not referenced by a warping operation.

13. The system of claim 1 , wherein each processing node also has an error correction module configured to correct artifacts.

14. The system of claim 13 , wherein the error correction module is also configured to perform color correction and color space transform on an output of the error correction module.

15. The system of claim 1 , wherein each processing node also has an interleaver module configured to transpose error corrected hogels to separate out individual bits per each hogel.

16. The system of claim 1 , wherein each processing node also has a pixel modulator configured to provide a pixel modulator output compatible with pixel input requirements of a light field display being used to display the reproduced light field.

17. A method for reproducing a light field for display from a compressed light field, the method comprising:
receiving, by a data decoder, data packets;
for each data packet,
determining, by the data decoder, whether a packet header of the data packet indicates that the data packet needs to be decoded, and
in response to determining that the packet header of the data packet indicates that the data packet needs to be decoded, decoding, by the data decoder, the data packet in accordance with a packet type; and

receiving, by an inverse image transform module, from the data decoder for blockwise seed and residual texture decoding and dequantization.

18. The method of claim 17 , further comprising using, by the inverse image transform module, a predefined image transform matrix, thereby allowing an inverse transform instruction to serve as a pipelined single instruction, multiple data instruction.

19. The method of claim 17 , further comprising receiving, by an interface module, the data packets and delivering the data packets to the data decoder.

20. The method of claim 17 , further comprising copying, by a hogel content repetition module, data between internal buffers having different lengths and different data widths without multiplicative scaling.

21. The method of claim 17 , further comprising receiving, by a forward depth-image-based rendering (DIBR) module, seed hogel disparity and producing warped disparity.

22. The method of claim 21 , further comprising reading, by a backward DIBR module, a generated temporary disparity from the forward DIBR module and calculating a current hogel address reference position in a seed hogel texture and generating hogels.

23. The method of claim 22 , further comprising using, by the backward DIBR module, residual disparity to combine with the seed hogel disparity to fix disparity errors.

24. The method of claim 17 , further comprising filling, by a backward depth-image-based rendering (DIBR) module, pixel positions not referenced by a warping operation.

25. The method of claim 17 , further comprising correcting by an error correction module artifacts.

26. The method of claim 25 , further comprising performing by the error correction module color correction and color space transform on an output of the error correction module.

27. The method of claim 17 , further comprising transposing by an interleaver module error corrected hogels to separate out individual bits per each hogel.

28. The method of claim 17 , further comprising providing by a pixel modulator a pixel modulator output compatible with pixel input requirements of a light field display being used to display the reproduced light field.

Assignees

• Ostendo Technologies Inc

Inventors

• Falkenberg Andreas
• Graziosi Danillo Bracco
• El-Ghoroury Hussein S

Classifications

• H04N19/436

  using parallelised computational arrangements · CPC title

• G03H1/0808

  Methods of numerical synthesis, e.g. coherent ray tracing [CRT], diffraction specific · CPC title

• H04N19/147

  according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) · CPC title

• H04N19/124

  Quantisation · CPC title

• H04N19/439

  using cascaded computational arrangements for performing a single operation, e.g. filtering · CPC title