Arbitrary visual features as fiducial elements

Patent metadata

Field	Value
Publication number	US-11775788-B2
Application number	US-202117246422-A
Country	US
Kind code	B2
Filing date	Apr 30, 2021
Priority date	May 15, 2019
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

Systems and methods for registering arbitrary visual features for use as fiducial elements are disclosed. An example method includes aligning a geometric reference object and a visual feature and capturing an image of the reference object and feature. The method also includes identifying, in the image of the object and the visual feature, a set of at least four non-colinear feature points in the visual feature. The method also includes deriving, from the image, a coordinate system using the geometric object. The method also comprises providing a set of measures to each of the points in the set of at least four non-colinear feature points using the coordinate system. The measures can then be saved in a memory to represent the registered visual feature and serve as the basis for using the registered visual feature as a fiducial element.

First claim

What is claimed is:

1. A computer-implemented method comprising:
registering a visual feature as a fiducial element, the registering including:
aligning a reference object and a visual feature;
capturing an image of the reference object and the visual feature;
identifying, in the image, a set of feature points in the visual feature;
deriving, from the image, a coordinate system using the reference object; and
providing a set of measures to each feature point of the set of feature points using the coordinate system; and

placing the registered visual feature in a locale.

2. The computer-implemented method of claim 1 , further comprising:
capturing an image of the visual feature in the locale using an imager; and
deriving, from the image of the visual feature in the locale and using the set of measures, a pose of the visual feature in the locale or the imager in the locale.

3. The computer-implemented method of claim 2 , further comprising:
generating, using data associated with the set of feature points, a unique identifier for the visual feature; and
determining, from the image of the visual feature in the locale, the unique identifier for the visual feature.

4. The computer-implemented method of claim 1 , wherein:
the reference object is a two-dimensional fiducial tag;
the visual feature is two-dimensional; and
the aligning the reference object and the visual feature is conducted by placing the two-dimensional fiducial tag and the visual feature proximate to each other on a surface.

5. The computer-implemented method of claim 4 , wherein capturing the image is conducted using a single visible light image capture device.

6. The computer-implemented method of claim 1 , further comprising:
identifying, in the image of the reference object and the visual feature, an alignment feature of the reference object and an alignment feature of the visual feature;
the reference object is a three-dimensional object;
the visual feature is three-dimensional; and
the aligning the reference object and the visual feature is conducted by placing the three-dimensional object and the visual feature: (i) proximate to each other; (ii) on a surface; and (iii) with the alignment feature of the three-dimensional object and the alignment feature of the visual feature located on a plane that is normal to the surface.

7. The computer-implemented method of claim 6 , wherein:
the capturing the image is conducted using a hero visible light camera and a witness visible light camera; and
the image includes depth information derived from the hero visible light camera and the witness visible light camera.

8. The computer-implemented method of claim 1 , wherein the visual feature is a two-dimensional picture with a nonrecurrent texture map.

9. The computer-implemented method of claim 1 , wherein the visual feature is a two-dimensional asymmetrical picture.

10. The computer-implemented method of claim 1 ,
the reference object is a regular two-dimensional array of two-dimensional fiducial tags;
the image consists of a set of pixels spaced according to a set of pixel receptors of an imager used to capture the image; and
the set of feature points are sub-pixel locations relative to the set of pixels.

11. The computer-implemented method of claim 1 ,
hashing data associated with the set of feature points to generate a unique identifier for the visual feature;
placing the visual feature in a locale;
capturing an image of the visual feature in the locale; and
identifying, from the image of the visual feature in the locale, the unique identifier for the visual feature.

12. The computer-implemented method of claim 11 , wherein the data associated with the set of feature points are Euclidean measures.

13. The computer-implemented method of claim 1 , wherein the visual feature is a first visual feature and the set of feature points is a first set of feature points, the method further comprising:
registering a second visual feature using the reference object to provide a second set of Euclidean measures to a second set of feature points using the coordinate system;
hashing data associated with the first set of feature points to generate a first unique identifier for the first visual feature;
hashing data associated with the second set of feature points to generate a second unique identifier for the second visual feature;
placing the first visual feature and the second visual feature in a locale;
capturing an image of both the first visual feature and the second visual feature in the locale; and
identifying, from the image of both the first visual feature and the second visual feature in the locale, the first unique identifier for the visual feature and the second unique identifier for the second visual feature.

14. The computer-implemented method of claim 1 , wherein the identifying is conducted using an automatic feature detector and a network trained to select points for maximum detectability.

15. The computer-implemented method of claim 1 , wherein the set of measures are distances to at least two other points in the set.

16. The computer-implemented method of claim 1 , wherein the set of measures are distances to a common point in the set.

17. The computer-implemented method of claim 1 , further comprising:
training a network to segment and identify the visual feature in any image;
placing the visual feature in a locale;
capturing an image of the visual feature in the locale; and
deriving, from the image of the visual feature in the locale, a pose of the visual feature in the locale using the set of measures and the network.

18. A non-transitory computer-readable medium storing instructions for executing a method comprising:
registering a visual feature as a fiducial element, the registering including:
aligning a reference object and a visual feature;
capturing an image of the reference object and the visual feature;
identifying, in the image, a set of feature points in the visual feature;
deriving, from the image, a coordinate system using the reference object; and
providing a set measures to each feature point of the set of feature points in the visual feature using the coordinate system; and

placing the registered visual feature in a locale.

Assignees

• Matterport Inc

Inventors

• Bradski Gary
• Amayeh Gholamreza
• Fathollahi Mona
• Rublee Ethan
• Vesom Grace
• Nguyen William

Classifications

• G06K7/1443 Primary

  locating of the code in an image · CPC title

• G06F18/2163

  Partitioning the feature space · CPC title

• G06K7/1417

  2D bar codes · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title