In situ creation of planar natural feature targets

Patent metadata

Field	Value
Publication number	US-10733798-B2
Application number	US-201313802834-A
Country	US
Kind code	B2
Filing date	Mar 14, 2013
Priority date	Mar 14, 2013
Publication date	Aug 4, 2020
Grant date	Aug 4, 2020

Abstract

Disclosed are a system, apparatus, and method for in-situ creation of planar natural feature targets. In one embodiment, a planar target is initialized from a single first reference image one or more subsequent images are processed. In one embodiment, the planar target is tracked in six degrees of freedom upon the processing of the one or more subsequent images and a second reference image is selected from the processed one or more subsequent images. In one embodiment, upon selecting the second reference image the planar target is refined to a more accurate planar target.

First claim

What is claimed is:

1. A processor-implemented method for planar target creation and tracking, the method comprising:
determining a planar target in a first reference image captured at a first camera pose of a camera, wherein the planar target includes one or more features;
initializing the planar target based on the first reference image;
displaying an augmented reality representation of the planar target based upon the initialization;
tracking the planar target as the camera moves from the first camera pose to a second camera pose;
displaying a distorted augmented reality representation of the planar target as the camera moves from the first camera pose;
selecting a second reference image based on an amount of movement or change in a camera pose from the first camera pose to the second camera pose meeting a threshold; and
displaying a more accurate augmented reality representation of the planar target based on the tracked planar target and the second reference image.

2. The processor-implemented method of claim 1 , further comprising:
assuming a fronto-parallel view of the planar target as a parameter for initializing the planar target.

3. The processor-implemented method of claim 2 , further comprising:
creating a rectified view of the planar target from the first reference image or the second reference image if neither the first reference image nor the second reference image are actual fronto-parallel views of the planar target.

4. The processor-implemented method of claim 1 , wherein initializing the planar target based on the first reference image further comprises assigning the entire first reference image as the planar target.

5. The processor-implemented method of claim 1 , wherein initializing the planar target based on the first reference image further comprises:
determining, for each feature of the planar target in the first reference image, a respective reference feature location within the planar target.

6. The processor-implemented method of claim 1 , wherein selecting the second reference image based upon the amount of movement or change in the camera pose from the first camera pose to the second camera pose meeting the threshold comprises:
measuring one or more translations, wherein each translation is measured between the first camera pose associated with the first reference image and a subsequent camera pose associated with one of one or more subsequent images; and
selecting one of the one or more subsequent images as the second reference image based upon the corresponding measured translation meeting the threshold.

7. The processor-implemented method of claim 1 , wherein displaying the more accurate augmented reality representation of the planar target based on the tracked planar target and the second reference image comprises:
extracting a plurality of reference features from the planar target in the first reference image;
extracting reference features from the second reference image;
matching reference from the second reference image to the plurality of reference features from the planar target in the first reference image;
triangulating matching reference features from the second reference image and from the planar target in the first reference image; and,
using the triangulation of the matching reference features to generate the more accurate augmented reality representation of the planar target.

8. The processor-implemented method of claim 1 , wherein the planar target is one or more of: partially occluded in the first reference image, of any planar shape, or in any orientation relative to gravity.

9. The processor-implemented method of claim 1 , wherein displaying the distorted augmented reality representation comprises displaying the distorted augmented reality representation prior to processing the second reference image.

10. The processor-implemented method of claim 1 , wherein displaying the more accurate augmented reality representation of the planar target comprises updating the augmented reality representation based on tracking the planar target in six degrees of freedom.

11. A non-transitory computer readable medium containing executable program instructions to configure a processor to:
determine a planar target in a first reference image captured at a first camera pose of a camera, wherein the planar target includes one or more features;
initialize the planar target based on the first reference image;
display an augmented reality representation of the planar target based upon the initialization;
track the planar target as the camera moves from the first camera pose to a second camera pose;
display a distorted augmented reality representation of the planar target as the camera moves from the first camera pose;
select a second reference image based on an amount of movement or change in a camera pose from the first camera pose to the second camera pose meeting a threshold; and
display a more accurate augmented reality representation of the planar target based on the tracked planar target and the second reference image.

12. The medium of claim 11 , wherein the instructions to initialize the planar target comprise instructions to configure the processor to:
assume a fronto-parallel view of the planar target as a parameter for initializing the planar target.

13. The medium of claim 12 , further comprising instructions to configure the processor to:
create a rectified view of the planar target from the first reference image or the second reference image if neither the first reference image nor the second reference image are actual fronto-parallel views of the planar target.

14. The medium of claim 11 , wherein the instructions to initialize the planar target based on the first reference image comprise instructions to configure the processor to assign the entire first reference image as the planar target.

15. The medium of claim 11 , wherein the instructions to initialize the planar target based on the first reference image comprise instructions to configure the processor to:
determine, for each feature of the planar target in the first reference image, a respective reference feature location within the planar target.

16. The medium of claim 11 , wherein the instructions to select the second reference image based upon the amount of movement or change in the camera pose from the first camera pose to the second camera pose meeting the threshold comprise instructions to configure the processor to:
measure one or more translations, wherein each translation is measured between the first camera pose associated with the first reference image and a subsequent camera associated with one of one or more subsequent images; and
select one of the one or more subsequent images as the second reference image based upon the corresponding measured translation meeting the threshold.

17. The medium of claim 11 , wherein the instructions to display the more accurate augmented reality representation of the planar target based on the tracked planar target and the second reference image comprise instructions to configure the processor to:
extract a plurality of reference features from the planar target in the first reference image;
extract reference features from the second reference image;
match reference from the second reference image to the plurality of reference features from the planar target in the first reference image;
triangulate matching reference features from the second reference image and from the planar target in the first reference image; and
use the triangulation of the matching reference features to generate t

Assignees

• Qualcomm Inc

Inventors

• Wagner Daniel
• Pan Qi

Classifications

• G06T7/74 Primary

  involving reference images or patches · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title