Image alignment method and image alignment system

Patent metadata

Field	Value
Publication number	US-9147118-B2
Application number	US-201313773596-A
Country	US
Kind code	B2
Filing date	Feb 21, 2013
Priority date	Feb 24, 2012
Publication date	Sep 29, 2015
Grant date	Sep 29, 2015

Abstract

An image alignment method includes steps of receiving a first image and a second image; scaling the first image and the second image by a ratio to generate a first downsized image and a second downsized image respectively; determining a first offset between the first downsized image and the second downsized image; selecting a first saliency region and a second saliency region from the first downsized image and the second downsized image; determining a second offset between a first sub-region within the first image and a second sub-region within the second image, the first sub-region and the second sub-region corresponding to the first saliency region and the second saliency region respectively; determining a final offset according to the ratio, the first offset and the second offset; and aligning the first image and the second image by the final offset.

First claim

What is claimed is:

1. An image alignment method, comprising steps of:
receiving a first image and a second image;
scaling the first image and the second image by a ratio to generate a first downsized image and a second downsized image respectively;
determining a first offset between the first downsized image and the second downsized image;
selecting a first saliency region and a second saliency region from the first downsized image and the second downsized image;
determining a second offset between a first sub-region within the first image and a second sub-region within the second image, the first sub-region and the second sub-region corresponding to the first saliency region and the second saliency region respectively;
determining a final offset according to the ratio, the first offset and the second offset; and
aligning the first image and the second image by the final offset;
wherein the final offset is determined by an equation as follows:
Df=D 1* R+D 2;

wherein Df represents the final offset, D 1 represents the first offset, R represents the ratio and D 2 represents the second offset.

2. The image alignment method of claim 1 , wherein the step of determining a first offset between the first downsized image and the second downsized image comprises steps of:
extracting a first feature map and a second feature map from the first downsized image and the second downsized image respectively;
comparing N reference areas and N candidate areas within the first feature map and the second feature map respectively to determine N first candidate offsets, wherein N is a positive integer; and
selecting a minimum one of the N first candidate offsets as the first offset.

3. The image alignment method of claim 2 , wherein the first feature map and the second feature map represent edge features of the first downsized image and the second downsized image respectively.

4. The image alignment method of claim 1 , wherein the step of selecting a first saliency region and a second saliency region from the first downsized image and the second downsized image comprises steps of:
aligning the first downsized image and the second downsized image by the first offset;
slicing the first downsized image into Q*R first blocks and slicing the second downsized image into Q*R second blocks, wherein Q and R are positive integers;
calculating Q*R gradient of sum of absolute differences between the Q*R first blocks and the Q*R second blocks horizontally and vertically;
selecting one of the Q*R first blocks having maximum gradient of sum of absolute differences as the first saliency region; and
shifting the first saliency region by the first offset to obtain the second saliency region.

5. The image alignment method of claim 1 , wherein the step of determining a second offset between a first sub-region and a second sub-region comprises steps of:
cropping the first sub-region and the second sub-region from the first image and the second image;
extracting a first sub-feature map and a second sub-feature map from the first sub-region and the second sub-region respectively;
comparing M first sub-areas of the first sub-feature map with M second sub-areas of the second sub-feature map to determine M second candidate offsets between the M first sub-areas and the M second sub-areas, wherein M is a positive integer; and
selecting a minimum one of the second candidate offsets as the second offset.

6. The image alignment method of claim 5 , wherein the first sub-feature map and the second sub-feature map represent edge features of the first sub-region and the second sub-region respectively.

7. An image alignment system, comprising:
an image input module for receiving a first image and a second image;
an image scaling module, coupled to the image input module and for scaling the first image and the second image by a ratio to generate a first downsized image and a second downsized image respectively;
an offset determination module, configured to determine a first offset between the first downsized image and the second downsized image, select a first saliency region and a second saliency region from the first downsized image and the second downsized image respectively, determine a second offset between a first sub-region within the first image and a second sub-region within the second image, the first sub-region and the second sub-region corresponding to the first saliency region and the second saliency region respectively, and determine a final offset according to the ratio, the first offset and the second offset; and
an image alignment module, configured to align the first image and the second image according to the final offset;
wherein the final offset is determined by an equation as follows:
Df=D 1* R+D 2;

wherein Df represents the final offset, D 1 represents the first offset, R represents the ratio and D 2 represents the second offset.

8. The image alignment system of claim 7 , further comprising a feature extraction module configured to extract a first feature map and a second feature map from the first downsized image and the second downsized image respectively; wherein the offset determination module is further configured to compare N reference areas and N candidate areas within the first feature map and the second feature map respectively to determine N first candidate offsets, and select a minimum one of the N first candidate offsets to be the first offset, wherein N is a positive integer.

9. The image alignment system of claim 8 , wherein the first feature map and the second feature map represent edge features of the first downsized image and the second downsized image respectively.

10. The image alignment system of claim 7 , wherein the offset determination module is further configured to align the first downsized image and the second downsized image by the first offset, slice the first downsized image into Q*R first blocks, slice the second downsized image into Q*R second blocks, calculate Q*R gradient of sum of absolute differences between the Q*R first blocks and the Q*R second blocks horizontally and vertically, select one of the Q*R first blocks, which is corresponding to the maximum one of the Q*R gradient of sum of absolute differences, to be the first saliency region, and shift the first saliency region by the first offset to obtain the second saliency region, wherein Q and R are positive integers.

11. The image alignment system of claim 7 , wherein the offset determination module is further configured to crop the first sub-region and the second sub-region from the first image and the second image, extract a first sub-feature map and a second sub-feature map from the first sub-region and the second sub-region respectively, compare M first sub-areas of the first sub-feature map with M second sub-areas of the second sub-feature map to determine M second candidate offsets between the M first sub-areas and the M second sub-areas, and select the minimum one of the second candidate offsets to be the second offset wherein M is a positive integer.

12. The image alignment system of claim 11 , wherein the first sub-feature map and the second sub-feature map represent edge features of the first sub-region and the second sub-region respectively.

13. An image alignment method, comprising steps of:
scaling a first image and a second image into a first downsized image and a second downsized image by a ratio;
determining a first offset between the first downsized image and the second downsized image;
cropping a first sub-region and a second sub-region from the first image and the second image respectively, wherein the first sub-region and the second s

Assignees

• Htc Corp

Inventors

• Lin Ching-Fu
• Chiu Chia-Wei
• Liang Kuo-Hsien

Classifications

• G06V10/24

  Aligning, centring, orientation detection or correction of the image · CPC title

• G06T7/33 Primary

  using feature-based methods · CPC title

• G06V10/752

  Contour matching · CPC title

• G06K9/6204

  Physics · mapped topic

• G06K9/32 Primary

  Physics · mapped topic