Mosaic picture generation

Patent metadata

Field	Value
Publication number	US-9253449-B2
Application number	US-201313772678-A
Country	US
Kind code	B2
Filing date	Feb 21, 2013
Priority date	Aug 23, 2010
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

A stability and, thus, also a quality of mosaic generation is improved by exploiting the advantages of global optimization while giving up performing the optimization fully on the basis of mutual alignment of the image information alone, i.e. on the basis of the result of a search for similarities, but when, additionally, information about error statistics of the capturing device, which took over portion-by-portion capturing of the object plane, is instead also taken into account in the positioning of the subimages in that a secondary constraint (secondary constraint) is set up for the capturing position variables in the optimization problem.

First claim

The invention clamed is:

1. An apparatus for generating a mosaic picture of an object plane, comprising:
a capturing device for portion-by-portion capturing of the object plane at a two-dimensional distribution of capturing positions by sequentially moving to the capturing positions along a capturing path using a motor, so as to acquire subimages which represent pictures of portions of the object plane which overlap, while allocating the capturing positions to the subimages, the capturing positions allocated to the subimages deviate from actual capturing positions in accordance with inaccuracies of the motor; and
a processor for determining offset vectors between pairs of overlapping subimages by a similarity analysis of the overlapping subimages and for solving an optimization problem for finding an optimum set of capturing position variables for the subimages for minimizing a measure of a deviation between the offset vectors of the pairs of overlapping subimages and differences of the capturing position variables of the pairs of overlapping subimages, while complying with a secondary constraint for the capturing position variables which depends on error statistics statistically describing a distribution of a deviation between offsets between pairs of capturing positions allocated to the subimages, to which the motor moves immediately following each other, and actual offsets between corresponding pairs of actual capturing positions,
wherein the secondary constraint limits, in dependence on the error statistics, the difference of capturing position variables for pairs of subimages which immediately follow one another along the capturing path.

2. The apparatus as claimed in claim 1 , wherein:
the two-dimensional distribution of capturing positions are substantially regularly arranged in rows and columns, and the capturing path is a zigzag path sequentially cycling through the two-dimensional distribution in rows of capturing positions, with traversing the rows of capturing positions sequentially along an advance direction aligned to the columns of capturing positions and alternatingly along a respective one of a first direction, a second direction, and the advance direction, so that capturing positions of subimages which immediately follow one another in accordance with the zigzag path are located, in relation to one another, along a respective one of the first direction, the second direction, and the advance direction, the error statistics being discriminative for the first direction, the second direction, and the advance direction, and
the secondary constraint for the differences of capturing position variables is different, in accordance with the error statistics, for pairs of subimages whose allocated capturing positions are located, in relation to one another, along the first direction, for pairs of subimages whose allocated capturing positions are located, in relation to one another, along the second direction, and for pairs of subimages whose allocated capturing positions are located, in relation to one another, along the advance direction.

3. The apparatus as claimed in claim 1 , wherein the processor solves, as the optimization problem, the quadratic optimization problem

arg
⁢

⁢
min

p

⁢

(



W
⁡

(

Ap
-
t

)



2

)

⁢

with the secondary constraint
Cp≦b,

wherein
p is a vector with 2×N components which correspond to the capturing position variables of the N subimages in a pairwise manner;
t is a vector with 2×M components which correspond to the M offset vectors in a pairwise manner;
A is a matrix with 2×M rows and 2×N columns, each of the rows comprising 2×N−2zeros and a one and a minus one at column positions in each case which correspond to the subimages between which that offset vector is determined which in the vector t corresponds to the component pair corresponding to the respective row;
W is a matrix with 2×M rows and 2×M columns;
C is a matrix with 2×N columns, each of the rows comprising 2×N−2 zeros and a one and a minus one at column positions in each case which correspond to the subimages which immediately follow each other along the capturing path; and
b is a vector whose components depend on the error statistics.

4. The apparatus as claimed in claim 3 , wherein W is a diagonal matrix whose diagonal components W ii in the respective row i of the matrix W correspond to a measure of a similarity of the subimages between which the offset vector at the corresponding component of the vector t is determined, in an overlap area of same as is defined by this very offset vector.

5. The apparatus as claimed in claim 1 , wherein the processor determines the error statistics by computing statistical measures of a distribution of the deviation between the offset vectors of the pairs of overlapping subimages and differences of the capturing positions of the pairs of overlapping subimages.

6. The apparatus as claimed in claim 1 , wherein the processor determines the error statistics by computing a central tendency and a dispersion of the deviations between the offset vectors of the pairs of overlapping subimages and differences of the capturing positions of the pairs of overlapping subimages.

7. The apparatus as claimed in claim 1 , wherein the two-dimensional distribution of capturing positions essentially corresponds to a regular two-dimensional distribution in columns and rows.

8. The apparatus as claimed in claim 1 , wherein the measure of the deviation is weighted with a measure of a similarity of the overlapping subareas for which the offset vectors are determined in each case.

9. The apparatus as claimed in claim 1 , wherein the capturing device comprises:
an image sensor;
optical device for imaging a field-of-view portion of the object plane onto the image sensor;
the motor; and
a controller for controlling the motor and the image sensor, so that the object plane is captured at the two-dimensional distribution of capturing points; wherein
the motor effects a lateral relative motion between the object plane, and the image sensor and the optical device.

10. The apparatus as claimed in claim 9 , wherein:
the controller:
dictates target capturing positions to the motor,
allocates the target capturing positions to the subimages as the capturing positions, and
instructs the image sensor, upon a confirmation signal for confirming that the target positions have been reached by the motor, to generate a respective subimage, and

the motor regulates a relative lateral location between the object plane, and the image sensor and the optical device, for such time until the target capturing positions have been reach

Assignees

• Fraunhofer Ges Forschung

Inventors

• Steckhan Dirk
• Zerfass Thorsten
• Wittenberg Thomas
• Elter Matthias
• Paulus Dietrich

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2200/32

  involving image mosaicing · CPC title

• G02B21/367 Primary

  providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison · CPC title

• G06T7/0034

  Physics · mapped topic

• H04N7/18 Primary

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title