Image noise reduction using lucas kanade inverse algorithm

Patent metadata

Field	Value
Publication number	US-9760997-B2
Application number	US-201615068976-A
Country	US
Kind code	B2
Filing date	Mar 14, 2016
Priority date	Mar 13, 2015
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

A reduced noise image can be formed from a set of images. One of the images of the set can be selected to be a reference image and other images of the set are transformed such that they are better aligned with the reference image. A measure of the alignment of each image with the reference image is determined. At least some of the transformed images can then be combined using weights which depend on the alignment of the transformed image with the reference image to thereby form the reduced noise image. By weighting the images according to their alignment with the reference image the effects of misalignment between the images in the combined image are reduced. Furthermore, motion correction may be applied to the reduced noise image.

First claim

The invention claimed is:

1. A method of transforming a first image to bring it closer to alignment with a second image, the method comprising:
implementing a multiple kernel tracking technique to determine positions of a set of candidate regions of the first image based on a similarity between a set of target regions of the second image and the set of candidate regions of the first image, wherein the target regions of the second image are respectively positioned over the positions of a predetermined set of points of the second image;
using at least some of the determined positions of the set of candidate regions to initialize a Lucas Kanade Inverse algorithm;
using the Lucas Kanade Inverse algorithm to determine a set of points of the first image which correspond to at least some of the predetermined set of points of the second image;
determining parameters of a transformation to be applied to the first image based on an error metric which is indicative of an error between a transformation of at least some of the determined set of points of the first image and the corresponding points of the predetermined set of points of the second image; and
applying the transformation to the first image to bring it closer to alignment with the second image.

2. The method of claim 1 wherein said implementing a multiple kernel tracking technique comprises iteratively optimizing the similarity between feature histograms of the set of target regions and corresponding feature histograms of the set of candidate regions by iteratively varying the positions of the candidate regions.

3. The method of claim 2 wherein said using at least some of the determined positions of the set of candidate regions to initialize a Lucas Kanade Inverse algorithm comprises discarding a candidate region if the feature histogram of the candidate region indicates that the candidate region is flat, wherein a discarded candidate region is not used to initialize the Lucas Kanade Inverse algorithm.

4. The method of claim 1 wherein said using the Lucas Kanade Inverse algorithm to determine a set of points of the first image which correspond to at least some of the predetermined set of points of the second image comprises, for each of the points of the set of points of the first image:
determining a warped version of an image patch surrounding the point; and
determining a Hessian matrix for the warped image patch which indicates a first sum of squared values of the gradients in the warped image in a first direction and a second sum of squared values of the gradients in the warped image in a second direction which is perpendicular to the first direction,
wherein the point is discarded if the ratio between the first and second sums of squared values of the gradients is greater than a threshold value or if the ratio between the second and first sums of squared values of the gradients is greater than the threshold value, wherein a discarded point is not used to determine the parameters of the transformation to be applied to the first image.

5. The method of claim 1 wherein the predetermined set of points of the second image are points of a uniform lattice.

6. The method of claim 1 wherein the first and second images are from a set of images, and wherein the method further comprises combining the transformed first image with the second image to form a reduced noise image.

7. The method of claim 6 wherein the second image is a reference image of the set of images.

8. The method of claim 6 wherein the set of images comprises either: (i) a plurality of images captured in a burst mode, or (ii) a plurality of frames of a video sequence.

9. A processing module for transforming a first image to bring it closer to alignment with a second image, the processing module comprising:
multiple kernel tracking logic comprising integrated circuitry configured to implement a multiple kernel tracking technique to determine positions of a set of candidate regions of the first image based on a similarity between a set of target regions of the second image and the set of candidate regions of the first image, wherein the target regions of the second image are respectively positioned over the positions of a predetermined set of points of the second image;
Lucas Kanade Inverse logic comprising integrated circuitry configured to use a Lucas Kanade Inverse algorithm to determine a set of points of the first image which correspond to at least some of the predetermined set of points of the second image, wherein the positions of at least some of the set of candidate regions determined by the multiple kernel tracking logic are used to initialize the Lucas Kanade Inverse algorithm; and
transformation logic comprising integrated circuitry configured to: (i) determine parameters of a transformation to be applied to the first image based on an error metric which is indicative of an error between a transformation of at least some of the determined set of points of the first image and the corresponding points of the predetermined set of points of the second image, and (ii) apply the transformation to the first image to bring it closer to alignment with the second image.

10. The processing module of claim 9 wherein the multiple kernel tracking logic is configured to implement the multiple kernel tracking technique by iteratively optimizing the similarity between feature histograms of the set of target regions and corresponding feature histograms of the set of candidate regions by iteratively varying the positions of the candidate regions.

11. The processing module of claim 10 wherein the multiple kernel tracking logic is configured to discard a candidate region if the feature histogram of the candidate region indicates that the candidate region is flat, wherein processing module is configured such that the Lucas Kanade Inverse logic does not use a discarded candidate region to initialize the Lucas Kanade Inverse algorithm.

12. The processing module of claim 9 wherein the Lucas Kanade Inverse logic is configured to use the Lucas Kanade Inverse algorithm to determine the set of points of the first image which correspond to at least some of the predetermined set of points of the second image by, for each of the points of the set of points of the first image:
determining a warped version of an image patch surrounding the point; and
determining a Hessian matrix for the image patch which indicates a first sum of squared values of the gradients in the warped image in a first direction and a second sum of squared values of the gradients in the warped image in a second direction which is perpendicular to the first direction,
wherein the Lucas Kanade Inverse logic is configured to discard the point if the ratio between the first and second sums of squared values of the gradients is greater than a threshold value or if the ratio between the second and first sums of squared values of the gradients is greater than the threshold value, wherein the Lucas Kanade Inverse logic is further configured to not use a discarded point to determine the parameters of the transformation to be applied to the first image.

13. The processing module of claim 9 wherein the predetermined set of points of the second image are points of a uniform lattice.

14. The processing module of claim 9 wherein the first and second images are from a set of images, and wherein the processing module further comprises combining logic configured to combine the transformed first image with the second image to form a reduced noise image.

15. The processing module of claim 14 wherein the second image is a reference image of the set of images.

Assignees

• Imagination Tech Ltd

Inventors

• Vivet Marc
• Brasnett Paul

Classifications

• H04N23/6845

  by combination of a plurality of images sequentially taken · CPC title

• G06T7/003 Primary

  Physics · mapped topic

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/337

  involving reference images or patches · CPC title

• G06T7/30 Primary

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title