Image registration using sliding registration windows

Patent metadata

Field	Value
Publication number	US-9247133-B2
Application number	US-201113151149-A
Country	US
Kind code	B2
Filing date	Jun 1, 2011
Priority date	Jun 1, 2011
Publication date	Jan 26, 2016
Grant date	Jan 26, 2016

Abstract

This disclosure pertains to devices, methods, and computer readable media for performing image registration. A few generalized steps may be used to carry out the image registration techniques described herein: 1) acquiring image data from an image sensor; 2) selecting a pair of overlapping image portions from the acquired image data for registration; 3) determining an area of “maximum energy” in one of the image portions being registered; 4) placing an image registration window over both image portions at the determined location of maximum energy; 5) registering the overlapping image portions using only the image data falling within the image registration windows; and 6) determining, according to one or more metrics, whether the image registration window should be shifted from a current location before registering subsequently acquired image portions.

First claim

What is claimed is:

1. An image registration method, comprising:
receiving a first image from an image sensor, the first image having a first number of rows, each row having a second number of pixels;
receiving a second image from the image sensor, the second image having the first number of rows, each row having the second number of pixels, wherein each row in the second image has a corresponding row in the first image and each pixel in the second image has a corresponding pixel in the first image;
identifying a subset of rows in the second image, wherein identifying a subset of rows in the second image comprises identifying a subset of contiguous rows, from among a plurality of subsets of contiguous rows in the second image, having a highest score, wherein each of the plurality of subsets of contiguous rows is determined by a location of an image registration window, wherein the image registration window slides one row at a time down a length of the second image, and wherein a score for each of the plurality of subsets of contiguous rows is based on a first number of pixel values in each row of the subset of contiguous rows; and
registering the first image and the second image using only a subset of pixels in each of the identified subset of rows in the second image and the corresponding subset of pixels in a corresponding subset of rows in the first image.

2. The method of claim 1 , wherein the first number of pixel values in each row of the subset of contiguous rows comprise less than all of the pixel values in the row.

3. The method of claim 1 , wherein identifying a subset of rows in the second image comprises:
determining a plurality of scores for the second image, wherein each score corresponds to a different subset of rows in the second image; and
identifying a subset of rows corresponding to a highest score.

4. The method of claim 3 , wherein determining a plurality of scores for the second image, wherein each score corresponds to a different subset of rows in the second image comprises, for each subset of rows:
determining a first value for each row in the subset of rows, wherein the first value is based on values of pixels in the row; and
combining the first value for each row in the subset of rows to generate a score for the subset of rows.

5. The method of claim 4 , wherein determining a first value for each row in the subset of rows comprises using values of pixels from less than all of the pixels in the row.

6. The method of claim 1 , wherein registering the first image and the second image comprises:
detecting first features in the subset of rows in the second image;
detecting second features in the corresponding subset of rows in the first image; and
registering the second image to the first image based on the first detected features and the second detected features.

7. The method of claim 1 , wherein registering the first image and the second image comprises:
cross-correlating the subset of pixels in the subset of rows in the second image with the corresponding subset of pixels in the corresponding subset of rows in the first image; and
registering the second image to the first image based on the cross-correlation.

8. A non-transitory program storage device having computer code stored thereon for causing a programmable control device to perform the method of claim 1 .

9. The non-transitory program storage device of claim 8 , wherein the computer program code further causes the programmable control device to perform the method of claim 2 .

10. A digital image capture device, comprising:
an image sensor;
a memory coupled to the image sensor; and
a program control device coupled to the memory, the program control device adapted to execute computer program instructions stored in the memory to perform the method of claim 1 .

11. The digital image capture device of claim 10 , wherein the computer program instructions further cause the program control device to perform the method of claim 2 .

12. An image registration method, comprising:
storing a first image in a first memory, the first image having a first number of rows of pixels, each row of pixels having a second number of pixels;
storing a second image in a second memory, the second image having the first number of rows of pixels, wherein each row of pixels in the second image has the second number of pixels, and wherein each pixel in the second image has a corresponding pixel in the first image;
identifying a first candidate portion of pixels in the second image from among a plurality of candidate portions in the second image, wherein each candidate portion comprises a subset of pixels from each of a contiguous subset of rows in the second image, wherein each of the plurality of subsets of contiguous rows is determined by a location of an image registration window, wherein the image registration window slides one row at a time down a length of the second image, wherein the first candidate portion of pixels is indicative of one or more features in the second image, and wherein the first candidate portion of pixels comprises a portion of pixels having a highest score from among the plurality of candidate portions in the second image;
identifying a first portion of pixels in the first image based on the first candidate portion of the second image, each pixel in the first portion of pixels in the first image having a corresponding pixel in the first candidate portion of pixels in the second image; and
registering the first image and the second image using only the first candidate portion of pixels in the second image and the corresponding pixels in the first portion of pixels in the first image.

13. The method of claim 12 , wherein identifying a first candidate portion of pixels in the second image further comprises:
(a) creating a first image registration window over the second image at a first location, the first image registration window comprising a predetermined number of rows of pixels in the second image, wherein the predetermined number of rows is less than the total number of rows in the second image;
(b) determining a value for each row of pixels in the first image registration window at the first location;
(c) determining a first score for the first image registration window at the first location based on the determined values for each of the rows of pixels in the first image registration window;
repeating steps (a)-(c) for each possible first location of the first image registration window over the second image;
identifying the first location of the first image registration window over the second image having the highest first score as the first maximum location; and
identifying the pixels falling within the identified first maximum location as the first candidate portion of pixels in the second image.

14. The method of claim 13 , wherein determining a value for each row of pixels comprises determining a contrast value.

15. The method of claim 14 , wherein determining the contrast value comprises determining the difference in luminance between the brightest pixel in the row of pixels and the darkest pixel in the row of pixels.

16. The method of claim 13 , wherein determining a value for each row of pixels comprises examining a subset of the total number of pixels in each row.

17. The method of claim 13 , further comprising:
storing a third image in a third memory, the third image having the first number of rows of pixels, wherein each row of pixels in the third image has the second number of pixels, and wherein each pixel in the third image has a corresponding pixel

Assignees

• Doepke Frank
• Sachs Todd S
• Hunter Kevin L
• Gatt Alexis
• Apple Inc

Inventors

• Doepke Frank
• Sachs Todd S
• Hunter Kevin L
• Gatt Alexis

Classifications

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N5/23238 Primary

  Electricity · mapped topic

• G06T7/0034

  Physics · mapped topic

• G06T3/0068

  Physics · mapped topic

• H04N5/2624

  for obtaining an image which is composed of whole input images, e.g. splitscreen · CPC title