Image stitching in the presence of a full field of view reference image

Patent metadata

Field	Value
Publication number	US-11832018-B2
Application number	US-202117771049-A
Country	US
Kind code	B2
Filing date	May 12, 2021
Priority date	May 17, 2020
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

Systems and methods for obtaining a seamless, high resolution, large field of view image comprise capturing a plurality of Tele images in a scene using a scanning Tele camera, each captured Tele image having an associated Tele field of view FOV T , retrieving a R image having a respective R image scene with a field of view greater than FOV T , analyzing the R image for defining an order of scanning positions according to which the folded Tele camera scans a scene to capture the plurality of Tele images, aligning the plurality of Tele images and the R image to obtain aligned Tele images, and composing the aligned Tele images into an output image. The output image may include at least parts of the R image and may be one of a stream of output images.

First claim

What is claimed is:

1. A method, comprising:
providing a folded Tele camera configured to scan and capture a plurality of Tele images, each captured image having a Tele image resolution (RES T ), a Tele image signal-to-noise-ratio (SNR T ) and a Tele field of view (FOV T );
obtaining and analyzing a reference (R) image with a R field of view FOV R >FOV T having a R image resolution RES R <RES T , and/or a R image with a signal-to-noise-ratio SNR R <SNR T ;
determining an order of one or more scanning FOV T positions for consecutive captures of the Tele images;
capturing a Tele image at each respective scanning FOV T position;
aligning the captured Tele images with segments of the R image to obtain aligned Tele images; and
using the aligned Tele images and the R image to create a new image having a field of view FOV N ≤FOV R , wherein the image resolution of the new image RES N >RES R and/or wherein the SNR of the new image SNR N >SNR R , wherein the determining an order of one or more scanning FOV T positions is performed so that the composed new image covers a maximal FOV according to a mechanical limitation of the scanning.

2. The method of claim 1 , further comprising aligning each Tele image with the R image immediately after its capture and prior to the capture of an immediately following Tele image, analyzing each Tele image for faults, and if faults are detected in the Tele image, re-capturing the Tele image at a same FOV T position, or, if faults are not detected in the Tele image, proceeding to capture an immediately following Tele image at a respective FOV T position.

3. The method of claim 2 , wherein the faults are selected from the group consisting of motion blur, electronic noise, rolling shutter, defocus blur and incorrect image alignment or obstructions.

4. The method of claim 2 , wherein the faults are mechanical faults.

5. The method of claim 1 , further comprising analyzing the aligned Tele images for faults, and if faults are detected in a particular Tele image, re-capturing the particular Tele image at a same FOV T position, or, if faults are not detected, using the aligned Tele images and the R image to create the new image.

6. The method of claim 1 , wherein the aligned Tele images and the R image are fed into an algorithm to create a super wide image having a field of view FOV SW , wherein a FOV segment within FOV R included in at least one FOV T of the captured Tele images has a field-of-view union-FOV T , and wherein union-FOV T <FOV SW ≤FOV R .

7. The method of claim 6 , wherein the determining an order of one or more scanning FOV T positions includes capturing an object in a Tele image with a specific FOV T to improve RES or SNR of a similar object included in FOV N but not included in the specific FOV T .

8. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions is performed so that the new image covers a region of interest selected by a user.

9. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions is performed so that the new image covers a region of interest defined by an algorithm.

10. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions is performed so that each Tele images includes a scene segments having a specific depth range or includes a scene segments that does not exceed a specific depth threshold.

11. The method of claim 1 , wherein the determining an order of one or more scanning FOV T positions is performed so that first moving objects are captured, and after the moving objects are captured, stationary objects are captured.

12. The method of claim 1 , wherein the determining an order of two or more FOV T positions is performed so that Tele images including specific scene characteristics within their respective FOV T s may be captured consecutively, and wherein the scene characteristics may be visual data such as texture or physical data such as brightness, depth or spectroscopic composition of a scene.

13. The method of claim 1 , wherein the folded Tele camera captures two or more Tele images at two or more respective FOV T positions within FOV R , wherein the determining an order of two or more scanning FOV T positions is performed so that a moving object is removed from a scene included in FOV R .

14. A method, comprising:
providing a folded Tele camera configured to scan and capture a plurality of Tele images, each captured image having a Tele image resolution (RES T ), a Tele image signal-to-noise-ratio (SNR T ) and a Tele field of view (FOV T );
obtaining and analyzing a reference (R) image with a R field of view FOV R >FOV T having a R image resolution RES R <RES T , and/or a R image with a signal-to-noise-ratio SNR R <SNR T ;
determining an order of one or more scanning FOV T positions for consecutive captures of the Tele images;
capturing a Tele image at each respective scanning FOV T position;
aligning the captured Tele images with segments of the R image to obtain aligned Tele images; and
using the aligned Tele images and the R image to create a new image having a field of view FOV N ≤FOV R , wherein the image resolution of the new image RES N >RES R and/or wherein the SNR of the new image SNR N >SNR R , and wherein the determining an order of two or more FOV T positions is performed so that capturing a minimal number of Tele images is required.

Assignees

• Corephotonics Ltd

Inventors

• Ilan Paz
• Vaisman Shai
• Katz Ruthy
• Teitel Adi
• Tzafrir Hagai
• Shabtay Gal
• Gigushinski Oded
• Azoulai Itamar

Classifications

• H04N23/10

  for generating image signals from different wavelengths · CPC title

• H04N5/2628 Primary

  Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation · CPC title

• H04N5/265 Primary

  Mixing · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title