Systems and methods for scale invariant 3D object detection leveraging processor architecture

Patent metadata

Field	Value
Publication number	US-9659217-B2
Application number	US-201615219798-A
Country	US
Kind code	B2
Filing date	Jul 26, 2016
Priority date	Aug 22, 2014
Publication date	May 23, 2017
Grant date	May 23, 2017

Abstract

An example method includes receiving a plurality of templates of a plurality of objects, where a template comprises feature values sampled at corresponding points of a two-dimensional grid of points positioned over a particular view of an object and scaled based on a depth of the object at the particular view. The method may further include receiving an image of an environment and determining a matrix representative of the image, where a row of the matrix comprises feature values sampled at a particular point of the two-dimensional grid positioned over one or more locations within the image and scaled based on depths of the one or more locations. The method may additionally include determining at least one similarity vector corresponding to at least one template and using the at least one similarity vector to identify at least one matching template for at least one object located within the image.

First claim

What is claimed is:

1. A method, comprising:
receiving, at a computing device, a plurality of templates of a plurality of objects, wherein a template of an object comprises a plurality of feature values sampled at corresponding points of a two-dimensional grid of points positioned over a particular view of the object and scaled based on a depth of the object at the particular view;
receiving, at the computing device, an image of an environment;
determining, by the computing device, feature values sampled at each point of the two-dimensional grid when the two-dimensional grid is positioned over a plurality of locations within the image of the environment and scaled based on respective depths of the plurality of locations within the image of the environment; and
identifying, by the computing device based on the determined feature values, a matching template from the plurality of templates for a particular object located at one of the plurality of locations within the image of the environment.

2. The method of claim 1 , wherein the image of the environment is received from an optical sensor, and wherein the two-dimensional grid is scaled relative to a canonical depth that is chosen based on a predicted viewing range of the optical sensor.

3. The method of claim 1 , wherein the plurality of feature values in each template of the plurality of templates comprise angles of two-dimensional gradient vectors representative of directional color changes.

4. The method of claim 1 , wherein the plurality of locations within the image comprise equally spaced pixels within the image.

5. The method of claim 1 , further comprising determining a matrix representative of the image, wherein a row of the matrix comprises the feature values sampled at a particular point of the two-dimensional grid positioned over the plurality locations within the image and scaled based on the respective depths of the plurality of locations within the image; and
using the matrix representative of the image to identify the matching template for the particular object.

6. The method of claim 5 , further comprising storing the matrix in a memory storage, wherein the rows of the matrix are stored linearly within the memory storage.

7. The method of claim 5 , wherein the rows of the matrix representative of the image are ordered based on a numbering of the points of the two-dimensional grid, wherein the numbering starts at a center point of the two-dimensional grid and increases according to a spiral of grid points extending from the center point.

8. The method of claim 7 , further comprising:
determining additional feature values from plurality of locations within the image at a plurality of additional grid points, wherein the plurality of additional grid points expand the two-dimensional grid; and
determining a plurality of additional rows for the matrix comprising the additional feature values, wherein the plurality of additional rows are ordered by numbering the plurality of additional grid points according to the spiral of grid points.

9. The method of claim 8 , further comprising:
identifying an object at a location from the plurality of locations within the image that is outside of the two-dimensional grid positioned over the location and scaled based on the depth of the location; and
adding the plurality of additional grid points to expand the two-dimensional grid to cover the identified object.

10. A non-transitory computer readable medium having stored therein instructions, that when executed by a computing system, cause the computing system to perform functions comprising:
receiving a plurality of templates of a plurality of objects, wherein a template of an object comprises a plurality of feature values sampled at corresponding points of a two-dimensional grid of points positioned over a particular view of the object and scaled based on a depth of the object at the particular view;
receiving an image of an environment;
determining feature values sampled at each point of the two-dimensional grid when the two-dimensional grid is positioned over a plurality of locations within the image of the environment and scaled based on respective depths of the plurality of locations within the image of the environment; and
identifying, based on the determined feature values, a matching template from the plurality of templates for a particular object located at one of the plurality of locations within the image of the environment.

11. The non-transitory computer readable medium of claim 10 , wherein the image of the environment is received from an optical sensor, and wherein the two-dimensional grid is scaled relative to a canonical depth that is chosen based on a predicted viewing range of the optical sensor.

12. The non-transitory computer readable medium of claim 10 , wherein the plurality of feature values in each template of the plurality of templates comprise numerical representations of normal vectors from surfaces of objects.

13. The non-transitory computer readable medium of claim 10 , wherein the plurality of locations within the image comprise equally spaced pixels within the image.

14. The non-transitory computer readable medium of claim 10 , the functions further comprising determining a matrix representative of the image, wherein a row of the matrix comprises the feature values sampled at a particular point of the two-dimensional grid positioned over the plurality locations within the image and scaled based on the respective depths of the plurality of locations within the image; and
using the matrix representative of the image to identify the matching template for the particular object.

15. The non-transitory computer readable medium of claim 14 , the functions further comprising storing the matrix in a memory storage, wherein the rows of the matrix are stored linearly within the memory storage.

16. The non-transitory computer readable medium of claim 14 , wherein the rows of the matrix representative of the image are ordered based on a numbering of the points of the two-dimensional grid, wherein the numbering starts at a center point of the two-dimensional grid and increases according to a spiral of grid points extending from the center point.

17. The non-transitory computer readable medium of claim 16 , the functions further comprising:
determining additional feature values from plurality of locations within the image at a plurality of additional grid points, wherein the plurality of additional grid points expand the two-dimensional grid; and
determining a plurality of additional rows for the matrix comprising the additional feature values, wherein the plurality of additional rows are ordered by numbering the plurality of additional grid points according to the spiral of grid points.

18. A system, comprising:
at least one optical sensor;
at least one processor; and
a non-transitory computer readable medium having stored therein instructions, that when executed by the at least one processor, cause the at least one processor to:
receive a plurality of templates of a plurality of objects, wherein a template of an object comprises a plurality of feature values sampled at corresponding points of a two-dimensional grid of points positioned over a particular view of the object and scaled based on a depth of the object at the particular view;
receive, from the at least one optical sensor, an image of an environment;
determine feature values sampled at each point of the two-dimensional grid when the two-dimensional grid is positioned over a plurality of locations within the image

Assignees

• X Dev Llc

Inventors

• Hinterstoisser Stefan

Classifications

• B25J5/007 Primary

  mounted on wheels · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/467

  Encoded features or binary features, e.g. local binary patterns [LBP] · CPC title

• B25J19/023

  including video camera means · CPC title