Arbitrary visual features as fiducial elements

Patent metadata

Field	Value
Publication number	US-10997448-B2
Application number	US-201916412784-A
Country	US
Kind code	B2
Filing date	May 15, 2019
Priority date	May 15, 2019
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

Systems and methods for registering arbitrary visual features for use as fiducial elements are disclosed. An example method includes aligning a geometric reference object and a visual feature and capturing an image of the reference object and feature. The method also includes identifying, in the image of the object and the visual feature, a set of at least four non-colinear feature points in the visual feature. The method also includes deriving, from the image, a coordinate system using the geometric object. The method also comprises providing a set of measures to each of the points in the set of at least four non-colinear feature points using the coordinate system. The measures can then be saved in a memory to represent the registered visual feature and serve as the basis for using the registered visual feature as a fiducial element.

First claim

What is claimed is:

1. A computer-implemented method comprising:
aligning a fiducial tag with a logo;
capturing, while the fiducial tag and the logo are aligned, an image of the fiducial tag and the logo;
identifying, in the image and using an automatic feature detector, a set of at least four non-colinear feature points in the logo;
deriving, from the image, a coordinate system using the fiducial tag; and
providing a set of measures to each of the points in the set of at least four non-colinear feature points using the coordinate system.

2. A computer-implemented method comprising:
aligning a geometric reference object and a visual feature;
capturing, while the geometric reference object and the visual feature are aligned, an image of the geometric reference object and the visual feature;
identifying, in the image of the geometric reference object and the visual feature, a set of at least four non-colinear feature points in the visual feature;
deriving, from the image of the geometric reference object and the visual feature, a coordinate system using the geometric object; and
providing a set of Euclidean measures to each of the points in the set of at least four non-colinear feature points using the coordinate system.

3. The computer-implemented method of claim 2 , further comprising:
placing the visual feature in a locale;
capturing an image of the visual feature in the locale using an imager; and
deriving, from the image of the visual feature in the locale and using the set of Euclidean measures, a pose of the visual feature in the locale or the imager in the locale.

4. The computer-implemented method of claim 3 , further comprising:
generating, using data associated with the set of at least four non-colinear feature points, a unique identifier for the visual feature; and
determining, from the image of the visual feature in the locale, the unique identifier for the visual feature.

5. The computer-implemented method of claim 2 , wherein:
the geometric reference object is a two-dimensional fiducial tag;
the visual feature is two-dimensional; and
the aligning step is conducted by placing the two-dimensional fiducial tag and the visual feature proximate to each other on a surface.

6. The computer-implemented method of claim 5 , wherein:
the capturing step is conducted using a single visible light camera.

7. The computer-implemented method of claim 2 , further comprising:
identifying, in the image of the geometric reference object and the visual feature, an alignment feature of the geometric reference object and an alignment feature of the visual feature;
the geometric reference object is a three-dimensional object;
the visual feature is three-dimensional; and
the aligning step is conducted by placing the three-dimensional object and the visual feature: (i) proximate to each other; (ii) on a surface; and (iii) with the first alignment feature of the three-dimensional object and the first alignment feature of the visual feature located on a plane that is normal to the surface.

8. The computer-implemented method of claim 7 , wherein:
the capturing step is conducted using a hero visible light camera and a witness visible light camera; and
the image includes depth information derived from the hero visible light camera and the witness visible light camera.

9. The computer-implemented method of claim 2 , wherein:
the visual feature is a two-dimensional picture with a nonrecurrent texture map.

10. The computer-implemented method of claim 2 , wherein:
the visual feature is a two-dimensional asymmetrical picture.

11. The computer-implemented method of claim 2 , wherein:
the geometric reference object is a regular two-dimensional array of two-dimensional fiducial tags;
the image consists of a set of pixels spaced according to a set of pixel receptors of an imager used to capture the image; and
the set of at least four non-colinear feature points are sub-pixel locations relative to the set of pixels.

12. The computer-implemented method of claim 2 , wherein:
hashing data associated with the set of at least four non-colinear feature points to generate a unique identifier for the visual feature;
placing the visual feature in a locale;
capturing an image of the visual feature in the locale; and
identifying, from the image of the visual feature in the locale, the unique identifier for the visual feature.

13. The computer-implemented method of claim 12 , wherein:
the data associated with the set of at least four non-colinear feature points is the Euclidean measures.

14. The computer-implemented method of claim 2 , wherein the visual feature is a first visual feature and the set of at least four non-colinear feature points is a first set of at least four non-colinear feature points, the method further comprising:
registering a second visual feature using the geometric reference object to provide a second set of Euclidean measures to a second set of at least four non-colinear points using the coordinate system;
hashing data associated with the first set of at least four non-colinear feature points to generate a first unique identifier for the first visual feature;
hashing data associated with the second set of at least four non-colinear feature points to generate a second unique identifier for the second visual feature;
placing the first visual feature and the second visual feature in a locale;
capturing an image of both the first visual feature and the second visual feature in the locale; and
identifying, from the image of both the first visual feature and the second visual feature in the locale, the first unique identifier for the visual feature and the second unique identifier for the second visual feature.

15. The computer-implemented method of claim 2 , wherein the identifying is conducted using an automatic feature detector and a network trained to select points for maximum detectability.

16. The computer-implemented method of claim 2 , wherein the Euclidean measures are distances to at least two other points in the set.

17. The computer-implemented method of claim 2 , wherein the Euclidean measures are distances to a common point in the set.

18. The computer-implemented method of claim 2 , further comprising:
training a network to segment and identify the visual feature in any image;
placing the visual feature in a locale;
capturing an image of the visual feature in the locale; and
deriving, from the image of the visual feature in the locale, a pose of the visual feature in the locale using the set of Euclidean measures and the network.

19. The computer-implemented method of claim 2 , wherein:
the geometric reference object is a two-dimensional fiducial tag; and
the visual feature is a logo.

20. A non-transitory computer-readable medium storing instructions for executing a method comprising:
aligning a geometric reference object and a visual feature;
capturing, while the geometric reference object and the visual feature are aligned, an image of the geometric reference object and the visual feature;
identifying, in the image of the geometric reference object and the visual feature, a set of at least four non-colinear feature points in the visual feature;
deriving, from the image of the geometric reference object and the visual feature, a coordinate system using the geometric object; and
providing a set of measures to each of the points in the set of at least four non-colinear feature points using the coordinate system.

Assignees

• Matterport Inc

Inventors

• Bradski Gary
• Amayeh Gholamreza
• Fathollahi Mona
• Rublee Ethan
• Vesom Grace
• Nguyen William

Classifications

• G06K7/1443 Primary

  locating of the code in an image · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title

• G06F18/2163

  Partitioning the feature space · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title