Object ingestion through canonical shapes, systems and methods

Patent metadata

Field	Value
Publication number	US-9501498-B2
Application number	US-201514623435-A
Country	US
Kind code	B2
Filing date	Feb 16, 2015
Priority date	Feb 14, 2014
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

An object recognition ingestion system is presented. The object ingestion system captures image data of objects, possibly in an uncontrolled setting. The image data is analyzed to determine if one or more a priori know canonical shape objects match the object represented in the image data. The canonical shape object also includes one or more reference PoVs indicating perspectives from which to analyze objects having the corresponding shape. An object ingestion engine combines the canonical shape object along with the image data to create a model of the object. The engine generates a desirable set of model PoVs from the reference PoVs, and then generates recognition descriptors from each of the model PoVs. The descriptors, image data, model PoVs, or other contextually relevant information are combined into key frame bundles having sufficient information to allow other computing devices to recognize the object at a later time.

First claim

What is claimed is:

1. An object recognition ingestion system comprising:
canonical shape database storing shape objects having geometrical attributes of canonical shapes, shape attributes, and having reference key frame points-of-view (PoVs); and
an object ingestion engine coupled with the canonical shape database and programmed to perform the steps of:
obtaining image data of at least one object;
deriving a set of edges related to the at least one object from the image data;
obtaining a shape result set from the canonical shape database where the shape result set include shape objects having shape attributes satisfying shape selection criteria determined as a function of geometrical information from the set of edges;
selecting at least one target shape object from shape objects in the shape result set;
generating an object model from the at least one target shape object and portions of the image data associated with the set of edges;
deriving a set of model key frame PoVs from the object model and the reference key frame PoVs associated with the at least one target shape object;
instantiating a descriptor object model from the object model, the descriptor model comprising recognition algorithm descriptors having locations on the object model relative to the model key frame PoVs;
creating a set of key frames bundles from the descriptor object model as a function of the set of model key frame PoVs; and
storing the set of key frame bundles in object recognition database.

2. The system of claim 1 , further comprising the object recognition database.

3. The system of claim 1 , wherein the shape objects stored in the canonical shape database include geometrical primitives.

4. The system of claim 3 , wherein at least one of the shape objects comprise a compound shape object comprising at least two geometric primitives.

5. The system of claim 3 , wherein the geometrical primitives include at least one of the following: a line, a square, a cube, a circle, a sphere, a cylinder, a cone, a box, a torus, a platonic solid, a triangle, a pyramid, and a box.

6. The system of claim 1 , wherein at least some of the shape objects stored in the canonical shape database represent 3D objects.

7. The system of claim 1 , wherein the shape objects stored in the canonical shape database comprises topological classifications.

8. The system of claim 1 , wherein the shape objects stored in the canonical shape database comprise object templates representing object classes.

9. The system of claim 8 , wherein the object templates include at least one of the following: a vehicle, a building, an appliance, a plant, a toy, a face, a person, and an internal organ.

10. The system of claim 1 , wherein the reference key frame PoVs comprise a normal vector.

11. The system of claim 1 , wherein the reference key frame PoVs comprise key frame PoV generation rules.

12. The system of claim 1 , wherein the image data comprises at least one of the following types of data: visible data, video data, video frame data, still image data, acoustic imaging data, medical image data, and game imaging data.

13. The system of claim 1 , wherein the geometrical attributes include at least one of the following: a length, a width, a height, a thickness, a radius, a diameter, an angle, a hole, a center, a formula, a texture, a bounding box, a chirality, a periodicity, an orientation, a pitch, and a number of sides.

14. The system of claim 1 , further comprising a mobile device that includes the object ingestion engine.

15. The system of claim 14 , wherein the mobile device further includes the canonical shape database.

16. The system of claim 14 , wherein the mobile device further includes the object recognition database.

17. The system of claim 1 , wherein the recognition ingestion engine is further programmed to perform the step of obtaining the shape result set as a function of edge descriptors associated with the set of edges.

18. The system of claim 17 , wherein the canonical shape database indexes the shape objects stored in the canonical shape database based on edge descriptors.

19. The system of claim 1 , wherein the recognition ingestion engine is programmed to perform the step of selecting the at least one target shape object based on a user selection.

20. The system of claim 1 , wherein the recognition ingestion engine is programmed to perform the step of selecting the at least one target shape object based on a score.

21. The system of claim 20 , wherein the score is determined as a function of at least one of the following: a location, a time, and a descriptor match.

22. The system of claim 1 , wherein the recognition algorithm descriptors include at least one of the following type of descriptors: SIFT, FREAK, FAST, DAISY, and BRISK.

23. The system of claim 1 , wherein at least one key frame bundle within the set of key frame bundles includes the following: a normal vector, an image, and a descriptor.

24. The system of claim 1 , further comprising an imaging sensor programmed to perform the step of capturing the image data of the at least one object.

Assignees

• Nant Holdings Ip Llc

Inventors

• Wnuk Kamil
• Mckinnon David
• Sudol Jeremi
• Song Bing
• Siddiqui Matheen

Classifications

• G06V20/46 Primary

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06T7/0085

  Physics · mapped topic

• G06K9/3241

  Physics · mapped topic

• G06K9/00201

  Physics · mapped topic

• G06K9/00744

  Physics · mapped topic