Image capturing device capable of blending images and image processing method for blending images thereof

Patent metadata

Field	Value
Publication number	US-9288392-B2
Application number	US-201313961897-A
Country	US
Kind code	B2
Filing date	Aug 8, 2013
Priority date	Feb 6, 2013
Publication date	Mar 15, 2016
Grant date	Mar 15, 2016

Abstract

An image capturing device and an image processing method are provided. The present method includes following steps. A first image and a second image are captured with a first focal length and a second focal length correspondingly. The motion corrected second image is produced by performing geometric correction procedure on the second image. A gradient operation is performed on each of the pixels of the first image to obtain a plurality of first gradients, and the gradient operation is performed on each of the pixels of the motion corrected second image to obtain a plurality of second gradients. The first gradients and the second gradients are compared and a first parameter map is generated according to the comparison results. A blending image is produced in according with the first parameter map and the first image, and an output image is produced at least in according with the blending image.

First claim

What is claimed is:

1. An image processing method, adapted to an image capturing device, the image processing method comprising:
capturing a first image with a first focal length, and capturing a second image with a second focal length, wherein the first focal length is focused on at least one main object;
performing a geometric calibration procedure on the second image to produce the motion calibrated second image;
performing a gradient operation on each pixel of the first image to produce a plurality of first gradients, and performing the gradient operation on each pixel of the motion calibrated second image to produce a plurality of second gradients;
comparing each of the first gradients and the corresponding second gradient to generate a plurality of first pixel comparison results, and generating a first parameter map according to the first pixel comparison results; and
producing a blending image according to the first parameter map and the first image, and producing an output image at least in accordance with the blending image,
wherein the step of comparing each of the first gradients and the corresponding second gradient to generate the first pixel comparison results, and generating the first parameter map according to the first pixel comparison results comprises:
dividing the second gradients by the corresponding first gradients to generate a plurality of gradient comparison values; and
generating a plurality of parameters according to the gradient comparison values, and recording the parameters as the first parameter map.

2. The image processing method as claimed in claim 1 , wherein the step of producing the output image at least in accordance with the blending image comprises:
capturing a third image with a third focal length;
performing the geometric calibration procedure on the third image to produce the motion calibrated third image;
performing the gradient operation on each pixel of the blending image to generate a plurality of third gradients, and performing the gradient operation on each pixel of the motion calibrated third image to generate a plurality of fourth gradients;
comparing each of the third gradients and the corresponding fourth gradient to generate a plurality of second pixel comparison results, and generating a second parameter map according to the second pixel comparison results; and
blending the motion calibrated third image and the blending image according to the second parameter map to produce the output image.

3. The image processing method as claimed in claim 1 , wherein the step of performing the geometric calibration procedure on the second image to produce the motion calibrated second image comprises:
performing a motion amount estimation on the first image and the second image to calculate a homography matrix; and
performing a geometric affine transformation on the second image according to the homography matrix, so as to obtain the motion calibrated second image.

4. The image processing method as claimed in claim 1 , wherein the step of generating the parameters according to the gradient comparison values comprises:
determining whether the gradient comparison values are greater than a first gradient threshold; and
setting the parameters corresponding to the gradient comparison values to a first value when the gradient comparison values are greater than the first gradient threshold.

5. The image processing method as claimed in claim 4 , wherein the step of generating the parameters according to the gradient comparison values comprises:
determining whether the gradient comparison values are greater than a second gradient threshold when the gradient comparison values are not greater than the first gradient threshold;
setting the parameters corresponding to the gradient comparison values to a second value when the gradient comparison values are greater than the second gradient threshold; and
setting the parameters corresponding to the gradient comparison values to a third value when the gradient comparison values are not greater than the second gradient threshold,
wherein the first gradient threshold is greater than the second gradient threshold.

6. The image processing method as claimed in claim 1 , wherein the step of producing the blending image according to the first parameter map and the first image comprises:
performing a blur procedure on the first image to generate a blur image; and
blending the first image and the blur image according to the first parameter map to produce a main object clear image.

7. The image processing method as claimed in claim 6 , wherein the step of blending the first image and the blur image according to the first parameter map to produce the main object clear image comprises:
determining whether the parameters are greater than a first blending threshold;
obtaining pixels of the blur image corresponding to the parameters to serve as pixels of the main object clear image when the parameters are greater than the first blending threshold;
determining whether the parameters are greater than a second blending threshold when the parameters are not greater than the first blending threshold;
calculating pixels of the main object clear image according to the parameters when the parameters are greater than the second blending threshold; and
obtaining pixels of the first image corresponding to the parameters to serve as pixels of the main object clear image when the parameters are not greater than the second blending threshold, wherein the first blending threshold is greater than the second blending threshold.

8. The image processing method as claimed in claim 1 , wherein the step of producing the blending image according to the first parameter map and the first image comprises:
calculating a plurality of sums of absolute differences corresponding to each pixel according to a pixel value of each of the pixels in the first image and the second image, and adjusting the parameters in the first parameter map according to the sums of absolute differences; and
blending the first image and the motion calibrated second image according to the adjusted first parameter map to generate a full depth of field image.

9. The image processing method as claimed in claim 8 , wherein the step of calculating the sums of absolute differences corresponding to each pixel according to the pixel value of each of the pixels in the first image and the second image and adjusting the parameters in the first parameter map according to the sums of absolute differences comprises:
determining a weighting factor of each of the parameters according to the sums of absolute differences when the sums of absolute differences are greater than a motion threshold, and adjusting the parameters according to the weighting factor, wherein each of the parameters decreases as the corresponding sum of absolute difference increases.

10. The image processing method as claimed in claim 8 , wherein the step of blending the first image and the motion calibrated second image according to the first parameter map to generate the full depth of field image comprises:
determined whether the parameters are greater than a first blending threshold;
obtaining pixels of the motion calibrated second image corresponding to the parameters to serve as pixels of the full depth of field image when the parameters are greater than the first blending threshold;
determining whether the parameters are greater than a second blending threshold when the parameters are not greater than the first blending threshold;
calculating pixels of the full depth of field image according to the parameters when the parameters are greater than the second blending threshold; and
obtaining pixels of the firs

Assignees

• Altek Semiconductor Corp

Inventors

• Chuang Che-Lun
• Chou Hong-Long

Classifications

• H04N23/68 Primary

  for stable pick-up of the scene, e.g. compensating for camera body vibrations · CPC title

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N5/23248 Primary

  Electricity · mapped topic

• H04N5/23229

  Electricity · mapped topic