Method for fast camera pose refinement for wide area motion imagery

Patent metadata

Field	Value
Publication number	US-9959625-B2
Application number	US-201514982030-A
Country	US
Kind code	B2
Filing date	Dec 29, 2015
Priority date	Dec 29, 2015
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

The present invention provides a method for fast, robust and efficient BA pipeline (SfM) for wide area motion imagery (WAMI). The invention can, without applying direct outliers filtering (e.g. RANSAC) or re-estimation of the camera parameters (e.g. essential matrix estimation) efficiently refine noisy camera parameters in very short amounts of time. The method is highly robust owing to its adaptivity with the persistency factor of each track. The present invention highly suitable for sequential aerial imagery, particularly for WAMI, where camera parameters are available from onboard sensors.

First claim

What is claimed is:

1. A method for camera pose refinement in three dimensional reconstruction of sequential frames of imagery of an image, comprising the steps of:
acquiring camera metadata, said metadata comprising camera position metadata and camera orientation metadata;
extracting interest points from each said image frame in said sequence;
comparing descriptors of said extracted interest points for each two successive image frames;
matching said descriptors so as to generate feature tracks;
generating a persistency factor for each said feature track as a function of said feature track's length;
computing a set of statistics for all said persistency factors when all feature tracks have been generated;
computing a triangulation based on said camera metadata and said feature tracks so as to generate estimated initial 3D interest points of said image;
weighting residuals generated from back projection error using said persistency factors;
computing an error function incorporating said weighted residuals, so as to reduce the effect of outlier noise; and
computing a bundle adjustment incorporating said weighted residuals and said camera metadata, so as to optimize said camera pose refinement and said estimated initial 3D interest points.

2. The method of claim 1 , wherein said set of statistics further comprises mean and standard deviation.

3. The method of claim 2 , wherein said mean is computed according to:

μ
=

1

N

3
⁢
D

⁢

∑

j
=
1

N

3
⁢
D

⁢

γ
j

where
γ j is a persistency factor;
μ is the mean of said persistency factor; and
N 3D is the number of 3D points.

4. The method of claim 3 , wherein said error function is computed according to

min

Ri
,
ti
,
Xj

⁢

∑

i
=
1

Nc

⁢

∑

j
=
1

N

3
⁢
D

⁢

(

γ
j

μ
+
σ

)

2

⁢

log
⁡

(

1
+

(

μ
+
σ

γ
j

)

2

⁢



x
ji

-

g
⁡

(

X
j

,

R
i

,

t
i

,

K
i

)



2

)

where
R i is a rotation matrix of an ith camera;
t i is a translation vector of an ith camera;
K i is a calibration matrix of an ith camera;
X j is a 3D point from said image;
x ji is an image coordinate in an ith camera
g(X j , R i , t i , K i ) is a projection model mapping of X j onto an image plane of an ith camera;
σ is the standard deviation of the lengths of all said feature tracks; and
N C is a number of cameras.

5. The method of claim 4 , wherein said bundle adjustment is computed according to:

min

Ri
,
ti
,
Xj

⁢

∑

i
=
1

Assignees

• The United States Of America As Represetned By The Secretary Of The Air Force
• Us Air Force

Inventors

• Seetharaman Gunasekaran
• Palaniappan Kannappan
• Akbarpour Hadi Ali

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/579 Primary

  from motion · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06K9/00208

  Physics · mapped topic