Fast initialization for monocular visual SLAM

Patent metadata

Field	Value
Publication number	US-9576183-B2
Application number	US-201313831405-A
Country	US
Kind code	B2
Filing date	Mar 14, 2013
Priority date	Nov 2, 2012
Publication date	Feb 21, 2017
Grant date	Feb 21, 2017

Abstract

Apparatuses and methods for fast visual simultaneous localization and mapping are described. In one embodiment, a three-dimensional (3D) target is initialized immediately from a first reference image and prior to processing a subsequent image. In one embodiment, one or more subsequent reference images are processed, and the 3D target is tracked in six degrees of freedom. In one embodiment, the 3D target is refined based on the processed the one or more subsequent images.

First claim

What is claimed is:

1. A processor-implemented method for visual simultaneous localization and mapping, the method comprising:
initializing a three-dimensional (3D) target based on a first set of target 3D points obtained from a first reference image, wherein the target 3D points in the first set are initialized along a plane at a predetermined initial depth value prior to processing a subsequent image;
processing one or more subsequent images;
tracking the 3D target in six degrees of freedom; and
refining the 3D target based on subsequent sets of target 3D points obtained from the processing of the one or more subsequent images, each subsequent set of target 3D points corresponding to one of the one or more subsequent images.

2. The processor-implemented method of claim 1 , further comprising:
displaying a correctly aligned, positioned, and distanced augmented reality representation of the 3D target upon initializing the 3D target and prior to the processing of the one or more subsequent images.

3. The processor-implemented method of claim 2 , further comprising:
updating the augmented reality representation of the 3D target while tracking the 3D target in six degrees of freedom.

4. The processor-implemented method of claim 1 , wherein the processing one or more subsequent images further comprises:
extracting 2D interest points from the one or more subsequent images.

5. The processor-implemented method of claim 1 , wherein initializing the 3D target further comprises:
extracting a reference set of 2D interest points from the first reference image;
determining the first set of target 3D points for the first reference image, each target 3D point in the first set corresponding to a 2D interest point in the reference set of 2D interest points; and
assigning the predetermined initial depth value, as a corresponding initial depth value, to each target 3D point.

6. The processor-implemented method of claim 5 , wherein the tracking the 3D target further comprises comparing a reference location corresponding to at least one of the target 3D points in the reference set to a corresponding updated 2D location of the at least one target 3D points extracted from the one or more subsequent images.

7. The processor-implemented method of claim 5 , wherein refining the 3D target further comprises:
determining an updated depth value for one or more of the target 3D points in the first set based on corresponding target 3D points in a subsequent set; and
replacing the assigned corresponding initial depth value for each of the one or more target 3D points in the first set with the corresponding updated depth value.

8. The processor-implemented method of claim 7 , further comprising:
determining when a threshold number of the target 3D points having updated depth values is met, wherein the threshold number of target 3D points is obtained from a corresponding subsequent image of the one or more subsequent images; and
assigning the respective subsequent image as a second reference image.

9. The processor-implemented method of claim 8 , wherein the refining of the 3D target further comprises performing a further refinement of the 3D target by triangulating the target 3D points with a plurality of 2D interest points extracted from the second reference image.

10. A computer readable non-transitory storage medium containing executable program instructions which cause a data processing device to perform a method for visual simultaneous localization and mapping, the method comprising:
initializing a three-dimensional (3D) target based on a first set of target 3D points obtained from a first reference image, wherein the target 3D points in the first set are initialized along a plane at a predetermined initial depth value prior to processing a subsequent image;
processing one or more subsequent images;
tracking the 3D target in six degrees of freedom; and
refining the 3D target based on the processing of subsequent sets of target 3D points obtained from the one or more subsequent images, each subsequent set of target 3D points corresponding to one of the one or more subsequent images.

11. The medium of claim 10 , further comprising:
displaying a correctly aligned, positioned, and distanced augmented reality representation of the 3D target upon initializing the 3D target and prior to the processing of the one or more subsequent images.

12. The medium of claim 11 , further comprising:
updating the augmented reality representation of the 3D target while tracking the 3D target in six degrees of freedom.

13. The medium of claim 10 , wherein the processing one or more subsequent images further comprises:
extracting 2D interest points from the one or more subsequent images.

14. The medium of claim 10 , wherein initializing the 3D target further comprises:
extracting a reference set of 2D interest points from the first reference image;
determining the first set of target 3D points, each target 3D point in the first set corresponding to a 2D interest point in the reference set of 2D interest points; and
assigning the predetermined initial depth value, as a corresponding initial depth value, to each target 3D point.

15. The medium of claim 14 , wherein the tracking the 3D target further comprises comparing a reference location corresponding to at least one of the target 3D points in the reference set to a corresponding updated 2D location of the at least one target 3D points extracted from the one or more subsequent images.

16. The medium of claim 14 , wherein refining the 3D target further comprises:
determining an updated depth value for one or more of the target 3D points in the first set based on corresponding target 3D points in a subsequent set; and
replacing the assigned corresponding initial depth value for each of the one or more target 3D points in the first set with the corresponding updated depth value.

17. The medium of claim 16 , further comprising:
determining when a threshold number of the target 3D points having updated depth values is met, wherein the threshold number of target 3D points is obtained from a corresponding subsequent image of the one or more subsequent images; and
assigning the respective subsequent image as a second reference image.

18. The medium of claim 17 , wherein the refining of the 3D target further comprises performing a further refinement of the 3D target by triangulating the target 3D points with a plurality of 2D interest points extracted from the second reference image.

19. A data processing device for visual simultaneous localization and mapping comprising:
a processor; and
a storage device coupled to the processor and configurable for storing instructions, which, when executed by the processor cause the processor to:
initialize a three-dimensional (3D) target based on a first set of target 3D points obtained from a first reference image, wherein the target 3D points in the first set are initialized along a plane at a predetermined initial depth value prior to processing a subsequent image;
process one or more subsequent images;
track the 3D target in six degrees of freedom; and
refine the 3D target based on the processing of subsequent sets of target 3D points obtained from the one or more subsequent images, each subsequent set of target 3D points corresponding to one of the one or more subsequent images.

20. The data processing device of claim 19 , further comprising instructions to:
display a correctly aligned, positioned, and distanced augmented reality r

Assignees

• Qualcomm Inc

Inventors

• Reitmayr Gerhard
• Mulloni Alessandro

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/0042

  Physics · mapped topic

• G06K9/00201 Primary

  Physics · mapped topic

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06V20/64 Primary

  Three-dimensional [3D] objects · CPC title