Motion estimation in real-time visual odometry system

Patent metadata

Field	Value
Publication number	US-2016110878-A1
Application number	US-201414515009-A
Country	US
Kind code	A1
Filing date	Oct 15, 2014
Priority date	Oct 15, 2014
Publication date	Apr 21, 2016
Grant date	—

Abstract

A motion determination system is disclosed. The system may receive a first camera image and a second camera image. The system may receive a first range image corresponding to the first camera image. The system may generate a first range map by fusing the first camera image and the first range image. The system may iteratively process a plurality of first features in the first range map to determine a change in position of the machine. The plurality of second features may correspond to the plurality of first features, and each of the plurality of first and second features is denoted by feature points in an image space of the camera.

First claim

What is claimed is:

1 . A motion determination system in a machine, comprising:
one or more memories storing instructions; and a controller configured to execute the instructions to perform operations including:
receiving a first camera image and a second camera image from a camera, wherein the first camera image is received earlier in time than the second camera image,
receiving, from a range detection unit, a first range image corresponding to the first camera image,
generating a first range map by fusing the first camera image and the first range image,
iteratively processing a plurality of first features in the first range map and a plurality of second features in the second camera image to determine a change in position of the machine,
wherein: the plurality of second features correspond to the plurality of first features, and each of the first features and second features are denoted by feature points in an image space of the camera.

2 . The system of claim 1 , wherein the range detection unit is a LIDAR, and the first and second range data are 3D point clouds.

3 . The system of claim 1 , wherein the iterative processing includes:
selecting a first number of iterations based on a desired robustness and a desired inlier ratio, wherein:
the inlier ratio is a ratio between a total number of inlier feature points from among the plurality of first feature points that fit an estimated model, and the plurality of first feature points, and
the estimated model describes a rotation and translation, respectively, between the first camera image and the second camera image.

4 . The system of claim 3 , wherein the iterative processing further includes, for each of the first number of iterations, the following operations:
selecting a plurality of third feature points from among the plurality of first feature points and a plurality of fourth feature points from among the plurality of second feature points, the plurality of fourth feature points corresponding to the plurality of third feature points, and determining a rotation matrix and a translation vector describing a rotation and translation, respectively, between the plurality of third feature points and the plurality of fourth feature points.

5 . The system of claim 4 , wherein the iterative further includes, for each of the first number of iterations, the following operations:
determining all inlier feature points for the first camera image by rotating and translating each of the plurality of first feature points using the determined rotation matrix and translation vector, and determining an inlier ratio for the iteration based on determined inlier feature points and the number of the plurality of first feature points.

6 . The system of claim 5 , wherein the iterative processing further includes:
executing each of the first number of iterations, determining, from among the first number of iterations, the iteration with the highest inlier ratio, and determining a final rotation matrix and a final translation vector based on the determined inlier feature points from the iteration with the highest inlier ratio.

7 . The system of claim 6 , wherein the iterative processing further includes:
determining whether the highest inlier ratio exceeds a predetermined threshold, in response to determining that the highest inlier ratio does not exceed a predetermined threshold:
selecting a second number of iterations that is greater than the first number of iterations, and
executing, for each of the second number of iterations, each of the steps executed for the first number of iterations.

8 . A computer-implemented method determining motion of a machine, the method comprising:
receiving a first camera image and a second camera image from a camera, wherein the first camera image is received earlier in time than the second camera image, receiving, from a range detection unit, a first range image corresponding to the first camera image, generating a first range map by fusing the first camera image and the first range image, iteratively processing a plurality of first features in the first range map and a plurality of second features in the second camera image to determine a change in position of the machine, wherein:
the plurality of second features correspond to the plurality of first features, and
each of the first features and second features are denoted by feature points in an image space of the camera.

9 . The method of claim 8 , wherein the range detection unit is a LIDAR, and the first and second range data are 3D point clouds.

10 . The method of claim 8 , wherein the iterative processing includes:
selecting a first number of iterations based on a desired robustness and a desired inlier ratio, and wherein:
the inlier ratio is a ratio between a total number of inlier feature points from among the plurality of first feature points that fit an estimated model, and the plurality of first feature points, and
the estimated model describes a rotation and translation, respectively, between the first camera image and the second camera image.

11 . The method of claim 10 , wherein the iterative processing further includes, for each of the first number of iterations, the following operations:
selecting a plurality of third feature points from among the plurality of first feature points and a plurality of fourth feature points from among the plurality of second feature points, the plurality of fourth feature points corresponding to the plurality of third feature points, and determining a rotation matrix and a translation vector describing a rotation and translation, respectively, between the plurality of third feature points and the plurality of fourth feature points.

12 . The method of claim 11 , wherein the iterative processing further includes, for each of the first number of iterations, the following operations:
determining all inlier feature points for the first camera image by rotating and translating each of the plurality of first feature points using the determined rotation matrix and translation vector, and determining an inlier ratio for the iteration based on determined inlier feature points and the number of the plurality of first feature points.

13 . The method of claim 12 , wherein the iterative processing further includes:
executing each of the first number of iterations, determining, from among the first number of iterations, the iteration with the highest inlier ratio, and determining a final rotation matrix and a final translation vector based on the determined inlier feature points from the iteration with the highest inlier ratio.

14 . The method of claim 13 , wherein the iterative processing further includes:
determining whether the highest inlier ratio exceeds a predetermined threshold, in response to determining that the highest inlier ratio does not exceed a predetermined threshold:
selecting a second number of iterations that is greater than the first number of iterations, and
executing, for each of the second number of iterations, each of the steps executed for the first number of iterations.

15 . A machine comprising:
a range detection unit; a camera; and a controller configured to execute instructions to perform operations including:
receiving a first camera image and a second camera image from the camera, wherein the first camera image is received earlier in time than the second camera image,
receiving, from the range detection unit, a first range image corresponding to the first camera image,
generating a first range map by fusing the first camera im

Assignees

• Caterpillar Inc

Inventors

• Chang Hong
• Friend Paul Russell

Classifications

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T7/20 Primary

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06K9/00791

  Physics · mapped topic

• G06K9/52

  Physics · mapped topic

• G06T2207/30244

  Camera pose · CPC title