Light-field camera

Patent metadata

Field	Value
Publication number	US-2016248987-A1
Application number	US-201615041818-A
Country	US
Kind code	A1
Filing date	Feb 11, 2016
Priority date	Feb 12, 2015
Publication date	Aug 25, 2016
Grant date	—

Abstract

The invention provides a light-field camera for capturing multiple views of a scene, representing samples of a light-field, having: a primary camera configured for capturing a primary digital two-dimensional image of the scene; a two-dimensional camera array having a plurality of secondary cameras, each of the secondary cameras being configured for capturing a secondary digital two-dimensional image of the scene to produce at least one set of secondary digital two-dimensional images of the scene; a semitransparent mirror arranged such that an incident light beam originating from the scene is split up in a first partial light beam, directed to the primary camera, and a second partial light beam, directed to the camera array; and a processing unit configured for receiving the primary digital two-dimensional image and the at least one set of secondary digital two-dimensional images and configured for computing depth information for the primary digital two-dimensional image or a digital two-dimensional image corresponding to the primary digital two-dimensional image based on the set of secondary digital two-dimensional images.

First claim

1 . A light-field camera for capturing multiple views of a scene, the multiple views representing samples of a light-field, the light-field camera comprising:
a primary camera configured for capturing a primary digital two-dimensional image of the scene; a two-dimensional camera array comprising a plurality of secondary cameras, wherein each of the secondary cameras is configured for capturing a secondary digital two-dimensional image of the scene in order to produce at least one set of secondary digital two-dimensional images of the scene; a semitransparent mirror arranged in such way that an incident light beam originating from the scene is split up in a first partial light beam, which is directed to the primary camera, and a second partial light beam, which is directed to the camera array; and a processing unit configured for receiving the primary digital two-dimensional image and the at least one set of secondary digital two-dimensional images and configured for computing depth information for the primary digital two-dimensional image or a digital two-dimensional image corresponding to the primary digital two-dimensional image based on the set of secondary digital two-dimensional images.

2 . The light-field camera according to claim 1 , wherein the primary camera fulfills at least one of the following conditions: comprising a higher resolution than the secondary cameras, comprising a better quality of color reproduction than the secondary cameras, comprising a wider dynamic range than the secondary cameras, comprising a larger signal-to-noise ratio than the secondary cameras, comprising higher bit depth than the secondary cameras.

3 . The light-field camera according to claim 1 , wherein a primary objective of the primary camera comprises fewer distortions than secondary objectives of the secondary cameras.

4 . The light-field camera according to claim 1 , wherein the processing unit is configured for computing the depth information for the digital two-dimensional output image corresponding to the primary digital two-dimensional image solely based on the set of secondary digital two-dimensional images and on the position of the primary camera relative to the secondary cameras.

5 . The light-field camera according to claim 1 , wherein the secondary cameras comprise substantially parallel optical axes.

6 . The light-field camera according to claim 1 , wherein the secondary cameras comprise the same optical and/or electrical properties.

7 . The light-field camera according to claim 1 , wherein the secondary cameras are arranged in a lattice-like manner.

8 . The light-field camera according to claim 1 , wherein the primary camera, the camera array and the semitransparent mirror are mounted to a rig.

9 . The light-field camera according to claim 7 , wherein the rig comprises a mechanical adjuster for adjusting a crossing point of an optical axis of the primary camera and the optical axis of one of the secondary cameras.

10 . The light-field camera according to claim 7 , wherein the rig comprises a mechanical adjuster for compensating a difference of an optical path length from the scene to the primary camera and an optical path length from the scene to the camera array.

11 . The light-field camera according to claim 7 , wherein the rig comprises a mechanical adjuster for adjusting an angular orientation of the primary camera with respect to a rotation about a first rotation axis parallel to an optical axis of the primary camera relative to an angular orientation of the camera array with respect to a rotation about an rotation axis parallel to the optical axes of the secondary cameras.

12 . The light-field camera according to claim 7 , wherein the rig comprises a mechanical adjuster for compensating a difference of an angle from the optical axis of the primary camera to the first partial light beam and an angle from the second partial light beam to the optical axes of one of the secondary cameras.

13 . The light-field camera according to claim 1 , wherein the secondary cameras are configured for capturing a plurality of sets of secondary digital two-dimensional images while the primary camera captures one primary digital two-dimensional image, wherein each set of secondary digital two-dimensional images comprises different exposure settings than the other sets of secondary digital two-dimensional images, and wherein the processing unit is configured for computing the depth information based on the plurality of sets of secondary digital two-dimensional images.

14 . The light-field camera according to claim 1 , wherein the secondary cameras are combined into groups, wherein each group is configured to produce a subset of the set of secondary digital two-dimensional images of the scene, wherein each secondary camera comprises the same exposure settings as the other secondary cameras of the same group but different exposure settings than the secondary cameras of the other groups.

15 . The light-field camera according to claim 1 , wherein the processing unit comprises a rectification module configured for simultaneously rectifying of the secondary digital two-dimensional images of the set of secondary digital two-dimensional images and the primary digital two-dimensional image by using feature points or checkerboards in order to produce a set of rectified secondary digital two-dimensional images and a rectified primary digital two-dimensional image.

16 . The light-field camera according to claim 14 , wherein the processing unit comprises a depth information computing module configured for computing a depth information for each of the secondary cameras based on the set of rectified secondary digital two-dimensional images.

17 . The light-field camera according to claim 15 , wherein the processing unit comprises a depth warping module configured for warping each of the depth information to a position of the rectified primary digital two-dimensional image in order to produce the depth information for the primary digital two-dimensional image or the digital two-dimensional image corresponding to the primary digital two-dimensional image.

18 . The light-field camera according to claim 1 , wherein the processing unit comprises a rectification module configured for rectifying of the secondary digital two-dimensional images of the set of secondary digital two-dimensional images by using feature points or checkerboards in order to produce a set of rectified secondary digital two-dimensional images.

19 . The light-field camera according to claim 17 , wherein the processing unit comprises a depth information computing module configured for computing a depth information for each of the secondary cameras based on the set of rectified secondary digital two-dimensional images.

20 . The light-field camera according to claim 18 , wherein the processing unit comprises a depth warping module configured for warping each of the disparity maps to a position of the primary digital two-dimensional image in order to produce the depth information for the primary digital two-dimensional image or the digital two-dimensional image corresponding to the primary digital two-dimensional image.

21 . The light-field camera according to claim 14 , wherein the processing unit comprises a depth information computing module configured for computing a depth information for each of the secondary cameras and the depth information for the primary camera based on the set of rectified secondary digital two-dimensional images and the rectif

Assignees

• Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E V

Inventors

• Zilly Frederik
• Keinert Joachim
• Ziegler Matthias
• Schoeberl Michael

Classifications

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/90 Primary

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/73

  by influencing the exposure time · CPC title