Methods, systems, and computer readable media for visual odometry using rigid structures identified by antipodal transform

Patent metadata

Field	Value
Publication number	US-9280832-B2
Application number	US-201514707632-A
Country	US
Kind code	B2
Filing date	May 8, 2015
Priority date	May 8, 2014
Publication date	Mar 8, 2016
Grant date	Mar 8, 2016

Abstract

The subject matter described herein includes methods for visual odometry using rigid structures identified by an antipodal transform. One exemplary method includes receiving a sequence of images captured by a camera. The method further includes identifying rigid structures in the images using an antipodal transform. The method further includes identifying correspondence between rigid structures in different image frames. The method further includes estimating motion of the camera based on motion of corresponding rigid structures among the different image frames.

First claim

What is claimed is:

1. A method for visual odometry using rigid structures identified by an antipodal transform, the method comprising:
receiving a sequence of images captured by a camera;
identifying rigid structures in the images using an antipodal transform;
identifying correspondence between rigid structures in different image frames; and
estimating a path of motion of the camera based on motion of corresponding rigid structures among the different image frames;
wherein identifying rigid structures using the antipodal transform includes:
generating a binary occupancy matrix for pixels in an image frame, wherein each element in the occupancy matrix indicates whether the pixel includes point on a structure or not;
linearizing the matrix;
marking occupied elements in the matrix as having a score that is different from unoccupied elements; and
for each unoccupied element in the matrix, determining a distance in each of a plurality of directions to the nearest occupied element, computing a score for each distance, and summing the scores for each distance, and identifying the rigid structures using the scores.

2. The method of claim 1 wherein identifying rigid structures using the scores includes identifying unoccupied elements with the highest scores as keypoints that comprise center points of rigid structures.

3. The method of claim 2 wherein identifying correspondence between rigid structures includes generating an image descriptor for each keypoint, the image descriptor including an encoded gradient magnitude, wherein the gradient magnitude represents a change in contrast between image pixels and neighboring image pixels.

4. The method of claim 3 wherein identifying correspondence between rigid structures includes comparing image descriptors in different image frames.

5. The method of claim 4 wherein comparing image descriptors includes utilizing a similarity metric to characterize differences between the image descriptors.

6. The method of claim 5 wherein the similarity metric comprises a Hamming distance.

7. The method of claim 6 wherein comparing the image descriptors includes maintaining sets of closest matching image descriptors among frame triplets after outliner elimination until a predetermined number of matching descriptors is located.

8. The method of claim 1 wherein estimating motion of the camera includes computing absolute rotation of the camera based on identified principal directions in the scene.

9. The method of claim 8 wherein the principal directions are identified using vanishing points computed as intersections of orthogonal groups of parallel lines in the scene.

10. The method of claim 9 comprising computing an absolute position of the camera in each frame.

11. The method of claim 10 wherein estimating motion of the camera includes estimating the motion based on the change in the absolute position of the camera between frames.

12. A system for visual odometry using rigid structures identified by an antipodal transform, the system comprising:
a line and point feature extractor for receiving a sequence of images captured by a camera, identifying rigid structures in the images using an antipodal transform, and identifying correspondence between rigid structures in different image frames; and
a camera motion estimator estimating a path of motion of the camera based on motion of corresponding rigid structures among the different image frames;
wherein the line and point feature extractor is configured to compute the antipodal transform by:
generating a binary occupancy matrix for pixels in an image frame, wherein each element in the occupancy matrix indicates whether the pixel includes point on a structure or not;
linearizing the matrix;
marking occupied elements in the matrix as having a score that is different from unoccupied elements;
for each unoccupied element in the matrix, determining a distance in each of a plurality of directions to the nearest occupied element, computing a score for each distance, and summing the scores for each distance, and identifying the rigid structures using the scores.

13. The system of claim 12 wherein the line and point feature extractor is configured to identify unoccupied elements with the highest scores as keypoints that comprise center points of rigid structures.

14. The system of claim 13 wherein the line and point feature extractor is configured to generate an image descriptor for each keypoint, the image descriptor including an encoded gradient magnitude, wherein the gradient magnitude represents a change in contrast between image pixels and neighboring image pixels.

15. The system of claim 14 wherein the line and point feature extractor is configured to compare image descriptors in different image frames.

16. The system of claim 15 wherein the line and point feature extractor is configured to compare image descriptors utilizing a similarity metric to characterize differences between the image descriptors.

17. The system of claim 16 wherein the similarity metric comprises a Hamming distance.

18. The system of claim 17 wherein the line and point feature extractor is configured to compare descriptors by maintaining sets of closest matching descriptors among frame triplets after outliner elimination until a predetermined number of matching image descriptors is located.

19. The system of claim 12 wherein the motion estimator is configured to estimate motion of the camera includes by computing absolute rotation of the camera based on identified principal directions in the scene.

20. The system of claim 19 wherein the principal directions are identified using vanishing points computed as intersections of orthogonal groups of parallel lines in the scene.

21. The system of claim 20 wherein the motion estimator is configured to compute an absolute position of the camera in each frame.

22. The system of claim 21 wherein the motion estimator is configured to estimate motion of the camera based on the change in the absolute position of the camera between frames.

23. A non-transitory computer readable medium having stored thereon executable instructions that when executed by the processor of a computer control the computer to perform steps comprising:
receiving a sequence of images captured by a camera;
identifying rigid structures in the images using an antipodal transform;
identifying correspondence between rigid structures in different image frames; and
estimating a path of motion of the camera based on motion of corresponding rigid structures among the different image frames;
wherein identifying rigid structures using the antipodal transform includes:
generating a binary occupancy matrix for pixels in an image frame, wherein each element in the occupancy matrix indicates whether the pixel includes point on a structure or not;
linearizing the matrix;
marking occupied elements in the matrix as having a score that is different from unoccupied elements; and

for each unoccupied element in the matrix, determining a distance in each of a plurality of directions to the nearest occupied element, computing a score for each distance, and summing the scores for each distance, and identifying the rigid structures using the scores.

Assignees

• Univ Pennsylvania

Inventors

• Cleveland Jonas
• Daniilidis Kostas

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/0042

  Physics · mapped topic

• G06T7/2033 Primary

  Physics · mapped topic

• G06T2207/30244

  Camera pose · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title