Image stitching in the presence of a full field of view reference image

Patent metadata

Field	Value
Publication number	US-12096150-B2
Application number	US-202318366849-A
Country	US
Kind code	B2
Filing date	Aug 8, 2023
Priority date	May 17, 2020
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

Systems and methods for obtaining a seamless, high resolution, large field of view image comprise capturing a plurality of Tele images in a scene using a scanning Tele camera, each captured Tele image having an associated Tele field of view FOV T , retrieving a R image having a respective R image scene with a field of view greater than FOV T , analyzing the R image for defining an order of scanning positions according to which the folded Tele camera scans a scene to capture the plurality of Tele images, aligning the plurality of Tele images and the R image to obtain aligned Tele images, and composing the aligned Tele images into an output image. The output image may include at least parts of the R image and may be one of a stream of output images.

First claim

What is claimed is:

1. A method, comprising:
providing a folded Tele camera configured to scan and capture a plurality of Tele images, each captured Tele image having a Tele image resolution (RES T ), a Tele image signal-to-noise-ratio (SNR T ) and a Tele field of view (FOV T );
obtaining and analyzing a reference (R) image with a R field of view FOV R >FOV T having a R image resolution RES R <RES T , and/or a R signal-to-noise-ratio SNR R <SNR T ;
determining an order of one or more scanning FOV T positions for consecutive captures of the Tele images;
capturing a Tele image at each respective scanning FOV T position;
immediately after the capture of a Tele image and prior to the capture of an immediately following Tele image at each respective scanning FOV T position aligning each captured Tele image with the R image to obtain aligned Tele images,
analyzing each Tele captured image for faults, and if faults are detected in the captured Tele image, re-capturing the Tele image at a same FOV T position, or, if faults are not detected in the Tele image, proceeding to capture an immediately following Tele image at a respective FOV T position; and
using the aligned Tele images and the R image to create a new image having a field of view FOV N ≤FOV R and FOV N >FOV T , wherein the image resolution of the new image RES N >RES R and/or wherein the SNR of the new image SNR N >SNR R .

2. The method of claim 1 , wherein the R image is a Wide image having a FOV W >FOV T , and wherein the Wide image is captured by a Wide camera included a multi-camera together with the folded Tele camera.

3. The method of claim 1 , wherein the new image is a super image.

4. The method of claim 1 , wherein the new image is a super wide image having a field of view FOV SW and wherein a FOV segment within FOV R included in at least one FOV T of the captured Tele images has a field-of-view union-FOV T , and wherein union-FOV T <FOV SW ≤FOV R .

5. The method of claim 1 , wherein the faults are selected from the group consisting of motion blur, electronic noise, rolling shutter, defocus blur and incorrect image alignment or obstructions.

6. The method of claim 1 , wherein the faults are mechanical faults.

7. The method of claim 1 , further comprising, by a user or an algorithm, selecting a size of FOV N and a position of FOV N within a scene included in FOV R .

8. The method of claim 1 , wherein the Tele camera has an effective focal length of 7-40 mm.

9. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions includes determining the order so that the new image covers a maximal FOV according to a mechanical limitation of the scanning.

10. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions includes determining the order so that the new image covers a region of interest defined by an algorithm.

11. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions includes determining the order so that the capturing of the Tele image at each respective scanning FOV T position includes capturing first moving objects, and then stationary objects in the captured Tele image.

12. The method of claim 1 , wherein the determining an order of one or more FOV T positions so includes determining the order such that the capturing of the Tele image at each respective scanning FOV T position includes capturing a minimal number of Tele images.

13. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions includes determining an order to follow a moving object with an object tracker.

14. The method of claim 1 , performed by a smartphone.

15. A method, comprising:
providing a folded Tele camera configured to scan and capture a plurality of Tele images, each captured Tele image having a Tele image resolution (RES T ), a Tele image signal-to-noise-ratio (SNR T ) and a Tele field of view (FOV T );
obtaining and analyzing a reference (R) image with a R field of view FOV R >FOV T having a R image resolution RES R <RES T , and/or a R signal-to-noise-ratio SNR R <SNR T ;
determining an order of one or more scanning FOV T positions for consecutive captures of the Tele images so that the new image covers a region of interest selected by a user;
capturing a Tele image at each respective scanning FOV T position;
aligning the captured Tele images with segments of the R image to obtain aligned Tele images;
using the aligned Tele images and the R image to create a new image having a field of view FOV N ≤FOV R and FOV N >FOV T , wherein the image resolution of the new image fulfills RES N >RES R and/or wherein the SNR of the new image fulfills SNR N >SNR R .

16. The method of claim 15 , wherein the R image is a Wide image having a FOV W , the Wide image captured by a Wide camera included a multi-camera together with the folded Tele camera.

17. The method of claim 15 , wherein the new image is a super image.

18. The method of claim 15 , wherein the new image is having a field of view FOV SW , and wherein a FOV segment within FOV R included in at least one FOV T of the captured Tele images has a field-of-view union-FOV T , and wherein union-FOV T <FOV SW ≤FOV R .

19. The method of claim 15 , wherein a user or an algorithm selects a size of FOV N and a position of FOV N within a scene included in FOV R .

20. The method of claim 15 , wherein the Tele camera has an effective focal length of 7-40 mm.

21. The method of claim 15 , performed by a smartphone.

Assignees

• Corephotonics Ltd

Inventors

• Ilan Paz
• Vaisman Shai
• Katz Ruthy
• Teitel Adi
• Tzafrir Hagai
• Shabtay Gal
• Gigushinski Oded
• Azoulai Itamar

Classifications

• H04N23/10

  for generating image signals from different wavelengths · CPC title

• H04N5/265 Primary

  Mixing · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title