3D multi-aperture imaging devices, multi-aperture imaging device, method for providing an output signal of a 3D multi-aperture imaging device and method for capturing a total field of view

Patent metadata

Field	Value
Publication number	US-10708570-B2
Application number	US-201816176248-A
Country	US
Kind code	B2
Filing date	Oct 31, 2018
Priority date	May 12, 2016
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

A 3D multi-aperture imaging device includes a plurality of image sensor areas. The 3D multi-aperture imaging device includes a first plurality of optical channels for projecting overlapping first partial fields of view of a total field of view on first image sensor areas of the image sensor and includes a second plurality of optical channels for projecting second partial fields of view of the total field of view overlapping each other and the first partial fields of view on second image sensor areas. The first and second pluralities of optical channels are arranged laterally offset from one another. The 3D multi-aperture imaging device includes a processor that is configured to receive image sensor data from the image sensor that is configured to provide an output signal including a data header, wherein the data header includes information regarding the structure of the 3D multi-aperture imaging device.

First claim

The invention claimed is:

1. A 3D multi-aperture imaging device, comprising:
an image sensor with a plurality of image sensor areas, wherein each image sensor area comprises a plurality of pixels;
a first plurality of optical channels for projecting overlapping first partial fields of view of a total field of view on first image sensor areas of the image sensor;
a second plurality of optical channels for projecting second partial fields of view of the total field of view overlapping each other and the first partial fields of view on second image sensor areas of the image sensor, wherein the first and second plurality of optical channels are arranged laterally offset from one another by a base distance;
a processor that is configured to receive image sensor data from the image sensor comprising information on the first and second partial fields of view projected on the first and second plurality of image sensor areas and that is configured to provide an output signal comprising a data header and payload data, wherein the data header comprises information regarding the structure of the 3D multi-aperture imaging device and wherein the payload data comprise image information acquired from the pixels of the first image sensor areas and the second image sensor areas;
wherein the processor is configured to form the output signal such that the data header comprises information regarding a pressure of an atmosphere surrounding the 3D multi-aperture imaging optics, an environmental temperature of the 3D multi-aperture imaging optics and/or an operating temperature of the 3D multi-aperture imaging optics; wherein the 3D multi-aperture imaging device is implemented according to at least one of:
that the processor is configured to form the output signal such that the data header comprises information regarding a number of the first plurality or the second plurality of optical channels;
that the processor is configured to form the output signal such that the data header comprises information regarding a segmentation of the total field of view into the first or second partial fields of view;
that the processor is configured to form the output signal such that the data header comprises information regarding the base distance;
that the processor is configured to form the output signal such that the data header comprises information regarding a defective pixel of the image sensor;
that the processor is configured to compress information on the first and second partial fields of view projected on the first and second plurality of image sensor areas to acquire compressed image data and to form the output signal such that the payload data comprise information regarding the compressed image data;
that the processor is configured to process information on the first and second partial fields of view projected on the first and second plurality of image sensor areas to acquire a total image, and wherein the processor is configured to form the output signal such that the payload data comprise information regarding the total image; and
that the data header or the payload data comprise, pixel by pixel or for pixel clusters, depth data, an indication on a segmentation of pixels towards pixel clusters and/or an indication of depth information and/or distance information regarding pixel clusters.

2. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding a number of the first plurality or the second plurality of optical channels.

3. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding a segmentation of the total field of view into the first or second partial fields of view.

4. The 3D multi-aperture imaging device according to claim 1 with a first image capturing module comprising the first plurality of optical channels and with at least one second image capturing module comprising the second plurality of optical channels, wherein the processor is configured to form the output signal such that the data header comprises information regarding a number of image capturing modules of the 3D multi-aperture imaging device.

5. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises, at least for a first image sensor area, information regarding a number of pixels concerning a first image extension direction and a number of pixels for a second image extension direction.

6. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises, for at least one of the optical channels, information regarding a spatial coordinate of the at least one optical channel in the 3D multi-aperture imaging device.

7. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding the base distance.

8. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding a field angle of at least one optical channel or that the data header comprises information regarding a central field area and an extension of the field along a first and second field of view extension direction.

9. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding a pixel size of at least one of the pixels of one of the first or second image sensor areas.

10. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding a distortion of at least one optical channel of the 3D multi-aperture imaging device.

11. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding vignetting of at least one optical channel of the 3D multi-aperture imaging device.

12. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the data header comprises information regarding a defective pixel of the image sensor.

13. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to form the output signal such that the payload data comprise information regarding a line break or a column break of the captured image.

14. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to compress information on the first and second partial fields of view projected on the first and second plurality of image sensor areas to acquire compressed image data and to form the output signal such that the payload data comprise information regarding the compressed image data.

15. The 3D multi-aperture imaging device according to claim 1 , wherein the processor is configured to process information on the first and second partial fields of view projected on the first and second plurality of image sensor areas to acquire a total image, and wherein the processor is configured to form the output signal such that the payload data comprise information regarding the total image.

Assignees

• Fraunhofer Ges Forschung
• Fraunhofer Ges Zur Foederung Der Angewandten Forschung E V

Inventors

• Wippermann Frank
• Brückner Andreas
• BRäUER ANDREAS

Classifications

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N13/239 Primary

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• H04N13/178 Primary

  Metadata, e.g. disparity information · CPC title

• H04N5/2258

  Electricity · mapped topic