Methods and systems for vision-based motion estimation

Patent metadata

Field	Value
Publication number	US-10339389-B2
Application number	US-201414476524-A
Country	US
Kind code	B2
Filing date	Sep 3, 2014
Priority date	Sep 3, 2014
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

Aspects of the present invention are related to methods and systems for vision-based computation of ego-motion.

First claim

What is claimed is:

1. A visual odometry method for estimating vehicle motion, said method comprising:
providing a vehicle equipped with a camera rigidly mounted and calibrated with respect to the vehicle;
moving the vehicle from a starting position in a global coordinate frame to a first location across a planar ground plane;
receiving an incoming image on the camera;
performing feature detection on said incoming image to identify a plurality of regions, wherein each region in said plurality of regions is associated with a key point in an incoming image coordinate frame;
selecting a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said incoming image coordinate frame;
performing feature matching between said plurality of feature descriptors for said incoming image coordinate frame and a plurality of feature descriptors selected for a previous image coordinate frame, thereby producing a plurality of feature correspondences;
for each feature correspondence in said plurality of feature correspondences, aligning key points from said previous image coordinate frame to said incoming image coordination frame;
projecting said key points to a previous world coordinate frame and an incoming world coordinate frame at the planar ground plane, thereby producing a plurality of pairs of world coordinates at the planar ground plane;
estimating vehicle motion from said plurality of pairs of world coordinates at the planar ground plane;
minimizing an accumulation of errors by selecting a key pose as follows:
comparing a rotation angle of said motion estimate to an angle threshold and a distance traveled by the vehicle to a distance threshold; and
when said angle comparison meets a first criterion or said distance comparison meets a second criterion;

projecting said key pose to a current camera pose in said global coordinate frame;
determining a motion trajectory from said current camera pose; and
updating said plurality of feature descriptors selected for a previous world coordinate frame to said plurality of feature descriptors for said incoming world coordinate frame, wherein
said motion estimate comprises a rotation matrix and a translation vector;
said rotation matrix is associated with no more than three degrees of freedom and an in-plane rotation that is in-plane with respect to the ground plane;
said motion estimate comprises a rigid transformation T k,k−1
where said rigid transformation T k,k−1 consists of the rotation matrix R k,k−1 between a previous time (k−1) and a current time (k), and the translation vector t k−k−1 between the previous time and the current time, such that

T

k
,

k
-
1

=

[

R

k
,

k
-
1

t

k
,

k
-
1

0

1

]

.

2. The method as described in claim 1 , wherein said performing feature detection comprises using a feature detector selected from the group consisting of an edge detector, a corner detector, a blob detector, a peak detector, a SIFT key-point detector, a SURF key-point detector, a FAST key-point detector, a grid-based FAST key-point detector, an ORB key-point detector, a MSER key-point detector, a BRIEF key-point detector, a BRISK key-point detector, a FREAK key-point detector and a STAR key-point detector.

3. The method as described in claim 1 , wherein said feature descriptor is a feature descriptor selected from the group consisting of a block of pixel values, a block of normalized pixel values, a block of gradient values, a block of adjusted pixel values, a SIFT feature descriptor, a SURF feature descriptor, an ORGB feature descriptor, a BRIEF feature descriptor, a BRISK feature descriptor and a FREAK feature descriptor.

4. The method as described in claim 1 , wherein said performing feature matching comprises a fast, nearest-neighbor search with a k-dimensional tree.

5. The method as described in claim 1 , wherein said performing feature matching comprises pruning a plurality of candidate matches.

6. The method as described in claim 1 , wherein said computing the motion estimate comprises using a motion estimator selected from the group consisting of an Orthogonal-Procrustes-Analysis-based motion estimator and an Absolute-Orientation-based motion estimator.

7. The method as described in claim 1 , wherein performing feature detection comprises performing feature detection using a first mathematical model; and,
wherein estimating vehicle motion comprises rejecting outliers not accounted for in said first mathematical model.

8. The method as described in claim 7 , wherein said rejecting outliers comprises performing RANSAC motion estimation.

9. The method as described in claim 1 , wherein:
said first criterion is said rotation angle is greater than said angle threshold; and
said second criterion is said distance traveled is greater than said distance threshold.

10. The method as described in claim 1 , wherein said generating a current camera pose comprises concatenating said estimated motion to a previous camera pose.

11. The method of claim 1 wherein estimating said motion using said rigid transformation T k,k−1 comprises minimizing the mean-squared-error (MSE), denoted as E, as follows:

E
=

1
n

⁢

∑

i
=

1
:
n

⁢



B
i
′

-

R

k

Assignees

• Sharp Laboratories America Inc

Inventors

• Xu Xinyu
• Liao Miao
• Van Beek Petrus J L

Classifications

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G05D1/0253

  extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow (determining position or orientation from images G06T7/70) · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title