Apparatus and a method for generating data representative of a pixel beam

Patent metadata

Field	Value
Publication number	US-12367601-B2
Application number	US-201716317206-A
Country	US
Kind code	B2
Filing date	Jul 11, 2017
Priority date	Jul 11, 2016
Publication date	Jul 22, 2025
Grant date	Jul 22, 2025

Abstract

There are several types of plenoptic devices having their proprietary file format. At present there is no standard supporting the acquisition and transmission of multi-dimensional information for an exhaustive over-view of the different parameters upon which a light-field depends. As such acquired light-field data for different cameras have a diversity of formats. The notion of pixel beam, which represents a volume occupied by a set of rays of light in an object space of an optical system of a camera is thus introduced. The method according to the invention enables to provide data representative of a collection of pixel beams describing a first optical system that is agnostic since these data are obtained by imaging the collection of pixel beams through a second optical system. Such data representative of a collection of pixel beams enable the generation of parameterized output images from which post-processing.

First claim

The invention claimed is:

1. A computer implemented method comprising:
receiving a first content captured by a first acquisition device comprising a plurality of pixels and having a first optical system, wherein the first content includes a first set of pixel beams each having at least one first associated parameter;
receiving a second content captured by a second acquisition device comprising a plurality of pixels and having a second optical system, wherein the second content includes a second set of pixel beams each having at least one second associated parameter;
wherein a pixel beam is a volume occupied by a set of light rays emitted in an object space of an optical system, the light rays passing through a pupil of the optical system and a conjugate of the pixel;
determining a projection surface as a virtual sensor having grid elements and an associated non-physical optical system, wherein the projection surface is located in an image space of the non-physical optical system;
computing the first set of pixel beams using intrinsic and extrinsic parameters of the first acquisition device and the projection surface to determine a first set of imaged pixel beams represented by conjugates of the first set of pixel beams in the non-physical optical system, wherein the first set of imaged pixel beams are imaged pixels beams which section is entirely located in a restitution pupil defined in the non-physical optical system;
computing the second set of pixel beams using intrinsic and extrinsic parameters of the second acquisition device and the projection surface to determine a second set of imaged pixel beams represented by conjugates of the second set of pixel beams in the non-physical optical system, wherein the second set of imaged pixel beams are imaged pixels beams which section is entirely located in the restitution pupil defined in the non-physical optical system;
wherein the first set of pixel beams and the second set of pixel beams are mixed together and form a unique set of pixel beams;
determining an intersection of the imaged pixel beams from the first and second sets of imaged pixel beams with the projection surface;
accumulating values of parameters associated with the imaged pixel beams from the first and second sets of imaged pixel beams based on an intersection of the imaged pixel beams with the grid elements on the projection surface, wherein the values of the parameters comprise at least one of radiance or color; and
generating an image of pixel values based on the accumulated values of the parameters.

2. The method of claim 1 , wherein the at least one first associated parameter is obtained by at least a sensor of the first acquisition device.

3. The method of claim 1 , wherein the first set of pixel beams are comprised of volumetric rays.

4. The method of claim 1 , wherein the non-physical optical system is a different optical system than the first optical system.

5. An apparatus comprising a processor configured to:
receive a first content captured by a first acquisition device comprising a plurality of pixels and having a first optical system, wherein the first content includes a first set of pixel beams each having at least one first associated parameter;
receive a second content captured by a second acquisition device comprising a plurality of pixels and having a second optical system, wherein the second content includes a second set of pixel beams each having at least one second associated parameter;
wherein a pixel beam is a volume occupied by a set of light rays emitted in an object space of an optical system, the light rays passing through a pupil of the optical system and a conjugate of the pixel;
determine a projection surface as a virtual sensor having grid elements and an associated non-physical optical system, wherein the projection surface is located in an image space of the non-physical optical system;
compute the first set of pixel beams using intrinsic and extrinsic parameters of the first acquisition device and the projection surface to determine a first set of imaged pixel beams represented by conjugates of the first set of pixel beams in the non-physical optical system, wherein the first set of imaged pixel beams are imaged pixels beams which section is entirely located in a restitution pupil defined in the non-physical optical system;
compute the second set of pixel beams using intrinsic and extrinsic parameters of the second acquisition device and the projection surface to determine a second set of imaged pixel beams represented by conjugates of the second set of pixel beams in the non-physical optical system, wherein the second set of imaged pixel beams are imaged pixels beams which section is entirely located in the restitution pupil defined in the non-physical optical system;
wherein the first set of pixel beams and the second set of pixel beams are mixed together and form a unique set of pixel beams;
determine an intersection of the imaged pixel beams from the first and second sets of imaged pixel beams with the projection surface;
accumulate values of parameters associated with the imaged pixel beams from the first and second sets of imaged pixel beams based on an intersection of the imaged pixel beams with the grid elements on the projection surface, wherein the values of the parameters comprise at least one of radiance or color; and
generate an image of pixel values based on the accumulated values of the parameters.

6. The apparatus of claim 5 , wherein the at least one first associated parameter is obtained by at least a sensor of the first acquisition device.

7. The apparatus of claim 5 , wherein the first set of pixel beams are comprised of volumetric rays.

8. The apparatus of claim 5 , wherein the non-physical optical system is a different optical system than the first optical system.

9. The method of claim 1 , wherein the grid elements on the projection surface are a pixel grid.

10. The method of claim 1 , further comprising accumulating values of parameters associated with the imaged pixel beams based on location within a restitution pupil defined in the non-physical optical system.

11. The apparatus of claim 5 , wherein the grid elements on the projection surface are a pixel grid.

12. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, perform the method of claim 1 .

Assignees

• Interdigital Ce Patent Holdings

Inventors

• Blonde Laurent
• Doyen Didier
• Boisson Guillaume

Classifications

• G06T2207/10052

  Images from lightfield camera · CPC title

• H04N13/106 Primary

  Processing image signals (for multi-view video sequence encoding H04N19/597) · CPC title

• G02B27/0955

  Lenses (lenses per se G02B3/00) · CPC title

• G02B27/0025

  for optical correction, e.g. distorsion, aberration · CPC title

• H04N13/243 Primary

  using three or more two-dimensional [2D] image sensors · CPC title