Point of view aberrations correction in a scanning folded camera

Patent metadata

Field	Value
Publication number	US-11910089-B2
Application number	US-202117633204-A
Country	US
Kind code	B2
Filing date	Jul 13, 2021
Priority date	Jul 15, 2020
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

Systems and methods for correcting point of view (POV) aberrations in scanning folded cameras and multi-cameras including such scanning folded cameras. In a Tele folded camera that includes an optical path folding element (OPFE) and an image sensor, the OPFE is tilted in one or two directions to direct the Tele folded camera towards a POV of a scene, a Tele image or a stream of Tele images is captured from the POV, the Tele image having POV aberrations and the POV aberrations are digitally corrected to obtain an aberration-corrected image or stream of images.

First claim

What is claimed is:

1. A method comprising:
a) providing a Tele folded camera that includes an optical path folding element (OPFE) and an image sensor;
b) tilting the OPFE by a rotation angle in one or more directions to direct the Tele folded camera towards a point of view (POV);
c) capturing a Tele image from the POV, the Tele image having a POV aberration in the shape of a non-rectangular tetragon introduced by the rotation angle of the OPFE; and
d) digitally correcting the Tele image having the POV aberration by applying to the Tele image having the POV aberration a geometric transformation related to the rotation angle of the OPFE to obtain a respective aberration-corrected image, and cropping and scaling the aberration-corrected image to obtain a respective aberration-corrected, cropped and scaled output image that has an output image center (OIC), an output image size and an output image width/height ratio.

2. The method of claim 1 , wherein the image sensor has an image sensor center, an active sensor width and an active sensor height, and wherein the OIC coincides with the image sensor center.

3. The method of claim 1 , wherein the OIC is selected so that a largest possible rectangular crop image size for a particular output image width/height ratio is achieved.

4. The method of claim 2 , wherein the OIC is located less than 10 pixel sizes from an ideal OIC, wherein the ideal OIC is where a particular image point is located at the OIC or an image center is located at the OIC.

5. The method of claim 2 , wherein the OIC is located less than a distance of 10% of the active sensor width from an ideal OIC, wherein the ideal OIC is where a particular image point is located at the OIC or an image center is located at the OIC.

6. The method of claim 1 , wherein a cropping criterion is selected such that an object-image magnification M of an object across different POVs varies from a constant value by less than 10%.

7. The method of claim 1 , wherein steps b and c are repeated to obtain a plurality of Tele images captured at a plurality of POVs, and wherein the OIC is selected such that a plurality of Tele images captured for all possible POVs cover a maximum rectangular area within a scene.

8. The method of claim 1 , wherein steps b and c are repeated to obtain a plurality of Tele images captured at a plurality of POVs, and wherein the OIC is selected such that a plurality of Tele images captured for a particular plurality of POVs cover a maximum rectangular area within a scene.

9. The method of claim 1 , wherein the geometric transformation uses calibration data captured during a camera calibration process.

10. The method of claim 1 , wherein the geometric transformation is a homography transformation.

11. A method, comprising:
a) providing a Tele folded camera that includes an optical path folding element (OPFE) and an image sensor;
b) tilting the OPFE in one or more directions to direct the Tele folded camera towards a plurality of points of view (POVs) of a calibration chart, each POV associated with a respective OPFE position;
c) capturing a respective Tele image of the calibration chart at each POV, each Tele image having a respective POV aberration in the shape of a non-rectangular tetragon introduced by the position of the OPFE, wherein the calibration chart includes location identifiers that allow to determine the POV for the given OPFE position from the respective Tele image, wherein the calibration chart includes angular identifiers that allow to determine the POV aberration for the given OPFE position from each Tele image;
d) from the respective Tele images of the calibration chart at each POV, deriving calibration data between each POV with its respective POV aberration and the respective OPFE position; and
e) using the calibration data to digitally correct the POV aberration of each Tele image by applying to the Tele image having the POV aberration a geometric transformation related to the position of the OPFE, thereby obtaining a respective POV aberration-corrected image.

12. The method of claim 11 , wherein the calibration chart is a checkerboard chart.

13. The method of claim 11 , wherein the calibration data is represented by a bi-directional function that assigns any OPFE position to a Tele POV and/or its respective POV aberration correction and vice versa.

14. The method of claim 13 , wherein the bi-directional function is a polynomial.

15. The method of claim 11 , wherein the calibration data is represented by a bi-directional Look-Up-Table that assigns any OPFE position to a Tele POV and/or its respective POV aberration correction and vice versa.

16. The method of claim 11 , wherein the calibration data is represented by a Look-Up-Table comprising a plurality of OPFE positions with associated values for Tele POVs and/or their respective POV aberration corrections.

17. The method of claim 16 , wherein the plurality of OPFE positions includes more than five OPFE positions.

18. The method of claim 16 , wherein the plurality of OPFE positions includes more than 100 OPFE positions.

19. The method of claim 11 , further comprising:
providing a Wide camera with a field of view FOV W larger than a field of view FOV T of the Tele folded camera;
between step d and e, in a first additional step, with a Tele image POV positioned within a respective Wide image FOV at a respective OPFE position associated with the Tele image POV, capturing an additional Tele image of the calibration chart along with capturing a Wide image of the calibration chart; and
in a second additional step, using the Tele and Wide images for deriving calibration data between the respective OPFE position, the Tele POV within the respective Wide FOV and the Tele image's POV aberration with respect to the Wide image.

20. The method of claim 19 , wherein the first and second additional steps are performed simultaneously.

21. The method of claim 19 , wherein all the steps are performed by a same operator.

22. The method of claim 19 , wherein steps a-d are performed by a first operator, and wherein the first and second additional steps are performed by a second operator.

23. The method of claim 19 , wherein the steps a-d are performed in a time frame of less than 10 s, and wherein the first and second additional steps are performed in a time frame of less than 10 s.

24. The method of claim 19 , wherein the first additional step does not include any additional image capture, and wherein the analysis and the deriving of the calibration data includes receiving external calibration data between the Tele folded camera and the Wide camera.

25. The method of claim 11 , further comprising:
providing a Wide camera with a field of view FOV W larger than a field of view FOV T of the Tele folded camera;
receiving external calibration data between the Tele folded camera and the Wide camera; and

using the external calibration data to digitally correct the POV aberration of each Tele image.

26. The method of claim 25 , wherein the external calibration data is represented by a bi-directional function that assigns any OPFE position to a Tele POV within the Wide FOV and/or the Tele image respective POV aberration correction with respect to the Wide image and vice versa.

27. The method of claim 25 , wherein the bi-directional function is a bi-directional polynomial.

28. The method

Assignees

• Corephotonics Ltd
• Corephotonics Lid

Inventors

• Azoulai Itamar
• Safra Gal
• Werbner Anat
• Yehudayi Ela
• Ilan Paz
• Geva Nadav
• Cohen Noy
• Shabtay Gal
• Goldenberg Ephraim
• Katz Ruthy

Classifications

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• G02B27/646

  compensating for small deviations, e.g. due to vibration or shake (movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68) · CPC title

• G03B17/12

  with means for supporting objectives, supplementary lenses, filters, masks, or turrets · CPC title

• G02B7/1805

  for prisms (G02B7/181 takes precedence) · CPC title