System and method for relating corresponding points in images with different viewing angles

Patent metadata

Field	Value
Publication number	US-9400939-B2
Application number	US-201414251636-A
Country	US
Kind code	B2
Filing date	Apr 13, 2014
Priority date	Apr 13, 2014
Publication date	Jul 26, 2016
Grant date	Jul 26, 2016

Abstract

A system, method and computer program product for relating corresponding points in images with an overlapping scene. An example method includes generating transformed images of a target image using different image transformations for each of transformed images. Texture descriptors are extracted for feature points in the transformed images and a reference image. Matched feature points are identified and inliers from matched feature points are selected. An aligning transformation is generated using the inliers for at least one of the transformed images. A panorama image is created with the target image and reference image after the images are aligned.

First claim

What is claimed is:

1. A computer implemented method for relating corresponding points in a plurality of images with an overlapping scene, the method comprising:
receiving a first image and a second image, the first image having a coordinate system;
detecting first feature points for the first image;
extracting first texture descriptors from the first feature points;
generating a plurality of transformed images of the second image using different image transformations for each of transformed images;
detecting second feature points for each of the transformed images;
extracting second texture descriptors from each of the transformed images;
determining a texture similarity between the first texture descriptors from the first image and the second texture descriptors from each of the transformed images;
identifying matched feature points from the first image and each of the transformed images with an associated texture similarity above a similarity threshold;
selecting inliers from matched feature points, the inliers are matched feature points that fit a model with a distance less than a distance threshold;
generating an aligning transformation using the inliers for at least one of the transformed images;
aligning the first image and the at least one of the transformed images using the aligning transformation such that the second image is transformed to the coordinate system to the first image; and
creating a panorama image with the first image and the transformed second image after the first image and the second image are aligned.

2. The method of claim 1 , further comprising selecting a matched image from the plurality of transformed images having a highest number of matched feature points above a match number threshold.

3. The method of claim 1 , further comprising normalizing image color in the first image and the second image.

4. The method of claim 1 , wherein generating the plurality of transformed images of the second image includes performing a set of quantized affine transformations of the second image.

5. The method of claim 1 , further comprising, for each of the transformed images:
calculating a coordinate range of the transformed image before generating the transformed image; and
applying translation factors so that all pixels in the transformed image have positive coordinates if pixels in the transformed images have negative coordinates.

6. The method of claim 1 , wherein determining the texture similarity between the first texture descriptors from the first image and the second texture descriptors from each of the transformed images includes performing bidirectional matching where the first texture descriptors from the first image are matched to the second texture descriptors from each of the transformed images, and the second texture descriptors from each of the transformed images are matched to the first texture descriptors from the first image.

7. The method of claim 1 , wherein detecting the first feature points and detecting the second feature points is performed using a Scale-Invariant Feature Transform (SIFT) algorithm.

8. The method of claim 1 , wherein detecting the first feature points and detecting the second feature points is performed using a Speeded Up Robust Features (SURF) algorithm.

9. A system for relating corresponding points in a plurality of images with an overlapping scene, the system comprising:
a first image having a coordinate system;
a second image;
a computer processor configured to:
detect first feature points for the first image;
extract first texture descriptors from the first feature points;
generate a plurality of transformed images of the second image using different image transformations for each of transformed images;
detect second feature points for each of the transformed images;
extract second texture descriptors from each of the transformed images;
determine a texture similarity between the first texture descriptors from the first image and the second texture descriptors from each of the transformed images;
identify matched feature points from the first image and each of the transformed images with an associated texture similarity above a similarity threshold;
select inliers from matched feature points, the inliers are matched feature points that fit a model with a distance less than a distance threshold;
generate an aligning transformation using the inliers for at least one of the transformed images; and
align the first image and the at least one of the transformed images using the aligning transformation such that the second image is transformed to the coordinate system of the first image; and

a panorama image with the first image and the second image after the first image and the second image are aligned.

10. The system of claim 9 , wherein the computer processor is further configured to normalize image color in the first image and the second image.

11. The system of claim 9 , wherein the computer processor is further configured to perform a set of quantized affine transformations of the second image.

12. The system of claim 9 , wherein the computer processor is further configured to, for each of the transformed images:
calculate a coordinate range of the transformed image before generating the transformed image; and
apply translation factors so that all pixels in the transformed image have positive coordinates if pixels in the transformed images have negative coordinates.

13. The system of claim 9 , wherein the computer processor is further configured to perform bidirectional matching where the first texture descriptors from the first image are matched to the second texture descriptors from each of the transformed images, and the second texture descriptors from each of the transformed images are matched to the first texture descriptors from the first image.

14. The system of claim 9 , wherein the computer processor is further configured to use a Scale-Invariant Feature Transform (SIFT) algorithm in detecting the first feature points and detecting the second feature points.

15. A computer program product for relating corresponding points in a plurality of images with an overlapping scene, the computer program product comprising:
a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code configured to:
receive a first image and a second image, the first image having a coordinate system;
detect first feature points for the first image;
extract first texture descriptors from the first feature points;
generate a plurality of transformed images of the second image using different image transformations for each of transformed images;
detect second feature points for each of the transformed images;
extract second texture descriptors from each of the transformed images;
determine a texture similarity between the first texture descriptors from the first image and the second texture descriptors from each of the transformed images;
identify matched feature points from the first image and each of the transformed images with an associated texture similarity above a similarity threshold;
select inliers from matched feature points, the inliers are matched feature points that fit a model with a distance less than a distance threshold;
generate an aligning transformation using the inliers for at least one of the transformed images;
align the first image and the at least one of the transformed images using the aligning transformation such that the second image is transformed to the coordinate system of the first image; and
create a panorama

Assignees

• IBM

Inventors

• Lin Chung-Ching
• Pankanti Sharathchandra U
• Smith John R

Classifications

• G06V10/7515 Primary

  Shifting the patterns to accommodate for positional errors · CPC title

• G06T7/40

  Analysis of texture (depth or shape recovery from texture G06T7/529) · CPC title

• G06K9/4671 Primary

  Physics · mapped topic

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06K9/52

  Physics · mapped topic