Robust tracking using point and line features

Patent metadata

Field	Value
Publication number	US-9406137-B2
Application number	US-201414278928-A
Country	US
Kind code	B2
Filing date	May 15, 2014
Priority date	Jun 14, 2013
Publication date	Aug 2, 2016
Grant date	Aug 2, 2016

Abstract

Disclosed embodiments pertain to apparatus, systems, and methods for robust feature based tracking. In some embodiments, a score may be computed for a camera captured current image comprising a target object. The score may be based on one or more metrics determined from a comparison of features in the current image and a prior image captured by the camera. The comparison may be based on an estimated camera pose for the current image. In some embodiments, one of a point based, an edge based, or a combined point and edge based feature correspondence method may be selected based on a comparison of the score with a point threshold and/or a line threshold, the point and line thresholds being obtained from a model of the target. The camera pose may be refined by establishing feature correspondences using the selected method between the current image and a model image.

First claim

What is claimed is:

1. A method of object tracking comprising:
computing a score for a camera captured current image comprising a target object, the score based, at least in part, on one or more metrics determined from a comparison of features in the current image and a prior image captured by the camera, and wherein the comparison is based on an estimated camera pose for the current image; and
selecting one of a point based, an edge based, or a combined point and edge based feature correspondence method based, at least in part, on a comparison of the score with at least one point threshold and at least one line threshold, the at least one point threshold and the at least one line threshold obtained from a model of the target.

2. The method of claim 1 , wherein the one or more metrics comprise at least one of:
a number of feature matches between the current image and the prior image;
a proportion of feature matches relative to an expected number of feature matches between the current image and the prior image; or
an average Normalized Cross Correlation (NCC) score for feature point matches between the current image and the prior image.

3. The method of claim 1 , wherein the model of the target comprises a plurality of stored images of the target object.

4. The method of claim 3 , further comprising:
determining feature correspondences between the current image and at least one model image using the selected feature correspondence method, the model image being selected based on the estimated camera pose, and
refining the estimated camera pose based on the feature correspondences obtained between the current image and the at least one model image by using a weighted least squares approximation.

5. The method of claim 3 , wherein:
each stored image of the target object comprises a plurality of image subdivisions, each image subdivision being associated with a count of point features and a count of edge features, and,
wherein the at least one point threshold and the at least one line threshold are obtained from a visible region of the model based, at least in part, on the count of point features and the count of edge features associated with each image subdivision in the visible region, wherein the visible region of the model corresponds to the current image.

6. The method of claim 5 , wherein the each image subdivision is a grid obtained by subdividing each image in the model into a plurality of grids.

7. The method of claim 5 , wherein the one or more metrics comprise:
a uniformity of a spatial distribution of point matches between the current and the prior image;
a number of good matches between the current and the prior image;
a number of bad matches between the current and the prior image;
a proportion of feature matches relative to an expected number of feature matches between the current image and the prior image; or
an average Normalized Cross Correlation (NCC) score for feature point matches between the current image and the prior image.

8. The method of claim 1 , wherein the estimated camera pose for the current image is obtained based, at least in part, on a prior camera pose obtained for the prior image and on an estimated location of the target object in the current image.

9. The method of claim 1 , wherein:
the at least one point threshold provides an indication of suitability of the model for point based feature tracking, and
the at least one line threshold provides an indication of suitability of the model for line based feature tracking.

10. A User device (UD) comprising:
a camera configured to capture a plurality of images of comprising a target object,
a memory configured to store a model of the target object, and
a processor coupled to the camera and the memory, the processor configured to:
compute a score for a current image captured by the camera, the score based, at least in part, on one or more metrics determined from a comparison of features in the current image and a prior image captured by the camera, and wherein the comparison is based on an estimated camera pose for the current image; and
select one of a point based, an edge based, or a combined point and edge based feature correspondence method based, at least in part, on a comparison of the score with at least one point threshold and at least one line threshold, the at least one point threshold and the at least one line threshold obtained from the model of the target object.

11. The UD of claim 10 , wherein the one or more metrics comprise at least one of:
a number of feature matches between the current image and the prior image;
a proportion of feature matches relative to an expected number of matches between the current image and the prior image; or
an average Normalized Cross Correlation (NCC) score for feature point matches between the current image and the prior image.

12. The UD of claim 10 , wherein the model of the target object comprises a plurality of stored images of the target object.

13. The UD of claim 12 , wherein the processor is further configured to:
determine feature correspondences between the current image and at least one model image using the selected feature correspondence method, the model image being selected based on the estimated camera pose, and
refine the estimated current camera pose based on the feature correspondences obtained between the current image and the at least one model image by using a weighted least squares approximation.

14. The UD of claim 12 , wherein:
each of the plurality of stored images of the target object comprises a plurality of image subdivisions, each image subdivision being associated with a count of point features and a count of edge features, and,
wherein the processor is further configured to:
obtain the at least one point threshold and the at least one line threshold from a visible region of the model based, at least in part, on the count of point features and the count of edge features associated with each image subdivision in the visible region, the visible region of the model corresponding to the current image.

15. The UD of claim 14 , wherein the each image subdivision is a grid obtained by subdividing each image in the model into a plurality of grids.

16. The UD of claim 14 , wherein metrics comprise:
a uniformity of a spatial distribution of feature point matches between the current image and the prior image;
a number of feature matches between the current image and the prior image;
a proportion of feature matches relative to an expected number of feature matches between the current image and the prior image; or
an average Normalized Cross Correlation (NCC) score for feature point matches between the current image and the prior image.

17. The UD of claim 10 , wherein the processor is further configured to:
determine the estimated camera pose for the current image based, at least in part, on a prior camera pose obtained for the prior image and on an estimated location of the target object in the current image.

18. The UD of claim 10 , wherein:
the at least one point threshold provides an indication of suitability of the model for point based feature tracking, and
the at least one line threshold provides an indication of suitability of the model for edge based feature tracking.

19. An apparatus comprising:
imaging means for capturing a plurality of images of comprising a target object;
storage means for storing a model of the target object, the storage means coupled to the imaging means;
means for computing a score for a current image c

Assignees

• Qualcomm Inc

Inventors

• Ahuja Dheeraj
• Kim Kiyoung
• Tsin Yanghai
• Najafi Shoushtari Seyed Hesameddin

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• H04N7/183 Primary

  for receiving images from a single remote source · CPC title

• G06T7/0044 Primary

  Physics · mapped topic

• G06T7/2033

  Physics · mapped topic