Device comprising a multi-aperture imaging device for accumulating image information

Patent metadata

Field	Value
Publication number	US-11330161-B2
Application number	US-202117351544-A
Country	US
Kind code	B2
Filing date	Jun 18, 2021
Priority date	Dec 21, 2018
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

A device includes a multi-aperture imaging device having an image sensor; an array of adjacently arranged optical channels, and a beam deflector for deflecting a beam path of the optical channels, wherein a relative position of the beam deflector is switchable between first and second positions so that in the first and second positions, the beam path is deflected towards first and second total fields of view, respectively. The device further includes controller adapted to control the beam deflector to move to the first and second positions to obtain imaging information of the first and second total fields of view, respectively, from the image sensor; and to insert a portion of the first imaging information into the second imaging information so as to obtain accumulated image information that in parts represents the first total field of view and in parts represents the second total field of view.

First claim

The invention claimed is:

1. A device comprising:
a multi-aperture imaging device including:
an image sensor;
an array of adjacently arranged optical channels, each optical channel having optics for projecting at least a partial field of view of a total field of view on an image sensor area of the image sensor; and
a beam deflector for deflecting a beam path of the adjacently arranged optical channels, wherein a relative location of the beam deflector is switchable between a first position and a second position, so that in the first position the beam path is deflected towards a first total field of view, and in the second position the beam path is deflected towards a second total field of view;
the device further comprising:
a controller adapted to control the beam deflector to move to the first position to acquire first imaging information of the first total field of view from the image sensor to control the beam deflector to move to the second position to acquire second imaging information of the second total field of view from the image sensor, and to insert a portion of the first imaging information into the second imaging information so as to acquire accumulated image information that in parts represents the first total field of view and in parts represents the second total field of view,
wherein the controller is adapted to create for the second imaging information a depth map comprising a plurality of depth planes and to insert the first imaging information in a predetermined depth plane of the second imaging information so as to acquire the accumulated image information, and wherein the predetermined depth plane corresponds, within a tolerance range of 10%, to a distance of the first total field of view from the device.

2. The device as claimed in claim 1 , wherein the first total field of view is arranged along a user device or along an oppositely arranged world direction of the device.

3. The device as claimed in claim 2 , wherein the second total field of view is arranged along a direction opposite to the first total field of view.

4. The device as claimed in claim 1 , wherein the controller is adapted to identify and segment a person in the first imaging information and to insert an image of the person into the second imaging information so as to acquire the accumulated image information.

5. The device as claimed in claim 4 , wherein the controller is adapted to automatically identify the person and to automatically insert the image of the person into the second imaging information.

6. The device as claimed in claim 4 , wherein the controller is adapted to identify and/or segment the portion while using the depth map generated by the device from the first imaging information.

7. The device as claimed in claim 1 , wherein the predetermined depth plane is based on a user input associated with a placement of the first imaging information in the second imaging information.

8. The device as claimed in claim 1 , wherein the controller is adapted to scale the first imaging information so as to acquire scaled first imaging information, and to insert the scaled first imaging information into the second imaging information so as to acquire the accumulated image information.

9. The device as claimed in claim 8 , wherein the controller is adapted to determine a distance of an object imaged in the first imaging information with respect to the device and to scale the first imaging information on a basis of a comparison of the determined distance with the predetermined depth plane in the second imaging information.

10. The device as claimed in claim 1 , wherein the controller is adapted to detect the first total field of view and the second total field of view within a time interval of 30 ms at the most.

11. The device as claimed in claim 1 , wherein the controller is adapted to acquire the accumulated image information as a video data stream.

12. The device as claimed in claim 1 , wherein the controller is adapted to acquire the accumulated image information as a single frame.

13. The device as claimed in claim 1 , wherein the first imaging information comprises an image of a user and the second imaging information comprises a world view of the device, wherein the controller is configured to segment the image of the user from the first imaging information and to insert it into the world view.

14. The device as claimed in claim 13 , wherein the controller is adapted to insert the image of the user into the world view in a manner that is correct in terms of depth.

15. The device as claimed in claim 10 , wherein the controller is adapted to detect a sequence of images of the first total field of view and/or of the second total field of view with different focal positions, and to create the depth map for the first total field of view and/or the second total field of view from the sequence of images.

16. A device comprising:
a multi-aperture imaging device including:
an image sensor;
an array of adjacently arranged optical channels, each optical channel having optics for projecting at least a partial field of view of a total field of view on an image sensor area of the image sensor; and
a beam deflector for deflecting a beam path of the adjacently arranged optical channels, wherein a relative location of the beam deflector is switchable between a first position and a second position, so that in the first position the beam path is deflected towards a first total field of view, and in the second position the beam path is deflected towards a second total field of view,

the device further comprising:
a controller adapted to control the beam deflector to move to the first position to acquire first imaging information of the first total field of view from the image sensor, to control the beam deflector to move to the second position to acquire second imaging information of the second total field of view from the image sensor, and to insert a portion of the first imaging information into the second imaging information so as to acquire accumulated image information that in parts represents the first total field of view and in parts represents the second total field of view,
wherein the controller is adapted to create for the second imaging information a depth map comprising a plurality of depth planes and to insert the first imaging information in a predetermined depth plane of the second imaging information so as to acquire the accumulated image information, wherein the controller is adapted to scale the first imaging information so as to acquire scaled first imaging information, and to insert the scaled first imaging information into the second imaging information so as to acquire the accumulated image information, and wherein the controller is adapted to determine a distance of an object imaged in the first imaging information with respect to the device and to scale the first imaging information on a basis of a comparison of the determined distance with the predetermined depth plane in the second imaging information.

17. A device comprising:
a multi-aperture imaging device including:
an image sensor;
an array of adjacently arranged optical channels, each optical channel having optics for projecting at least a partial field of view of a total field of view on an image sensor area of the image sensor; and
a beam deflector for deflecting a beam path of the adjacently arranged optical channels, wherein a relative location of the beam deflector is switchable between a first position and a second position, so that in the first position the beam path is deflected towards a first total fie

Assignees

• Fraunhofer Ges Forschung

Inventors

• Wippermann Frank
• Brückner Andreas
• Duparré Jacques

Classifications

• H04N5/2621 Primary

  Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability · CPC title

• H04N23/45 Primary

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/58

  Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title