Image processing system and method for calculating transformation parameters to register a first image to a second image

Patent metadata

Field	Value
Publication number	US-9508117-B1
Application number	US-201514792945-A
Country	US
Kind code	B1
Filing date	Jul 7, 2015
Priority date	Jul 7, 2015
Publication date	Nov 29, 2016
Grant date	Nov 29, 2016

Abstract

It is often desirable to register a first image to a second image, such as to form a panoramic image. The image registration technique discussed herein forms first and second gradients of the first and second images, respectively, then aligns phase vectors of the first and second gradients by estimating the parameters of a projective (homographic) coordinate transformation that can map the first gradient to the second gradient. The estimated parameters can be used to map the first image to the second image. In some examples, each gradient pixel includes a complex number, such as a unit vector, having a normalized amplitude and a phase vector that indicates the direction of greatest change, at that pixel, for the respective image. Aligning the image gradient phase vectors, rather than image intensity values, can align images produced under different lighting conditions, and/or produced in different wavelength regions of the electromagnetic spectrum.

First claim

What is claimed is:

1. An image processing system for calculating transformation parameters to register a first image to a second image, the first and second images representing at least partially overlapping portions of a scene, the system comprising:
one or more processors; and
memory including instructions that, when executed on the one or more processors, configure the one or more processors to:
receive data corresponding to the first and second images;
calculate, from the received data, first and second gradients of the first and second images, respectively, the first and second gradients having respective phases that indicate a direction of greatest change in the first and second images, respectively; and
calculate, from the first and second gradients, estimated parameters of a coordinate transformation that maps the first gradient to the second gradient, the estimated parameters being calculated using spatiotemporal derivatives and without relying on features in the first or second images.

2. The image processing system of claim 1 , wherein the one or more processors are further configured to calculate the first and second gradients iteratively, beginning with reduced-resolution versions of the first and second images, and ending with full-resolution versions of the first and second images.

3. The image processing system of claim 2 , wherein each iteration increases a number of pixels along one dimension by a factor of 2.

4. The image processing system of claim 1 , wherein the instructions, when executed on the one or more processors, configure the one or more processors to further:
apply a projective mapping to the first image, based on the estimated parameters, to form a third image, the third image being registered to the second image; and
stitch the second and third images together to form a stitched image.

5. The image processing system of claim 1 , wherein the instructions, when executed on the one or more processors, configure the one or more processors to further:
apply a projective mapping to the first image, based on the estimated parameters, to form a third image, the third image being registered to the second image; and
combine the second and third images together to form a high-resolution image having a higher resolution than the first and second images.

6. The image processing system of claim 1 , wherein the instructions, when executed on the one or more processors, configure the one or more processors to further:
calculate, based at least in part on the estimated parameters, an estimate of a difference in camera position and orientation between the first and second images.

7. The image processing system of claim 1 , wherein the instructions, when executed on the one or more processors, configure the one or more processors to further:
apply a projective mapping to the first image, based on the estimated parameters, to form a third image, the third image being registered to the second image;
subtract a linear combination of the second and third images to form a difference image;
apply a threshold to the difference image to form a thresholded difference image; and
evaluate the thresholded difference image to locate one or more objects in the scene that moved between a time at which the first image was captured and a time at which the second image was captured.

8. A method for calculating transformation parameters to register a first image to a second image, the first and second images representing at least partially overlapping portions of a scene, the method comprising using one or more processors to:
receive data corresponding to first and second images;
calculate, from the received data, first and second gradients of the first and second images, respectively, the first and second gradients having respective phases that indicate a direction of greatest change in the first and second images, respectively; and
calculate, from the first and second gradients, estimated parameters of a coordinate transformation that maps the first gradient to the second gradient, the estimated parameters being calculated using spatiotemporal derivatives and without relying on features in the first or second images.

9. The method of claim 8 , wherein the one or more processors are further configured to calculate the first and second gradients iteratively, beginning with reduced-resolution versions of the first and second images, and ending with full-resolution versions of the first and second images.

10. The method of claim 9 , wherein each iteration increases a number of pixels along one dimension by a factor of 2.

11. The method of claim 9 , further comprising using the one or more processors to further:
apply a projective mapping to the first image, based on the estimated parameters, to form a third image, the third image being registered to the second image; and
stitch the second and third images together to form a stitched image.

12. The method of claim 9 , further comprising using the one or more processors to further:
apply a projective mapping to the first image, based on the estimated parameters, to form a third image, the third image being registered to the second image; and
combine the second and third images together to form a high-resolution image having a higher resolution than the first and second images.

13. The method of claim 9 , further comprising using the one or more processors to further:
calculate, based at least in part on the estimated parameters, an estimate of a difference in camera position and orientation between the first and second images.

14. The method of claim 9 , further comprising using the one or more processors to further:
apply a projective mapping to the first image, based on the estimated parameters, to form a third image, the third image being registered to the second image;
subtract a linear combination of the second and third images to form a difference image;
apply a threshold to the difference image to form a thresholded difference image; and
evaluate the thresholded difference image to locate one or more objects in the scene that moved between a time at which the first image was captured and a time at which the second image was captured.

15. A computer-readable storage medium storing instructions executable by one or more processors of an image processing system, the instructions to configure the one or more processors to:
receive data corresponding to first and second images;
calculate, from the received data, first and second gradients of the first and second images, respectively, the first and second gradients having respective phases that indicate a direction of greatest change in the first and second images, respectively; and
calculate, from the first and second gradients, estimated parameters of a coordinate transformation that maps the first gradient to the second gradient, the estimated parameters being calculated using spatiotemporal derivatives and without relying on features in the first or second images.

16. The computer-readable storage medium of claim 15 , wherein the one or more processors are further configured to calculate the first and second gradients iteratively, beginning with reduced-resolution versions of the first and second images, and ending with full-resolution versions of the first and second images.

17. The computer-readable storage medium of claim 16 , wherein each iteration increases a number of pixels along one dimension by a factor of 2.

18. The computer-readable storage medium of claim 15 , wherein the instructions, when executed on the one

Assignees

• Raytheon Co

Inventors

• Caber Timothy Edward

Classifications

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T3/0087 Primary

  Physics · mapped topic

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T3/4061

  by injecting details from different spectral ranges · CPC title

• G06T7/0036

  Physics · mapped topic