Object ingestion through canonical shapes, systems and methods

Patent metadata

Field	Value
Publication number	US-10832075-B2
Application number	US-201816123764-A
Country	US
Kind code	B2
Filing date	Sep 6, 2018
Priority date	Feb 14, 2014
Publication date	Nov 10, 2020
Grant date	Nov 10, 2020

Abstract

An object recognition ingestion system is presented. The object ingestion system captures image data of objects, possibly in an uncontrolled setting. The image data is analyzed to determine if one or more a priori know canonical shape objects match the object represented in the image data. The canonical shape object also includes one or more reference PoVs indicating perspectives from which to analyze objects having the corresponding shape. An object ingestion engine combines the canonical shape object along with the image data to create a model of the object. The engine generates a desirable set of model PoVs from the reference PoVs, and then generates recognition descriptors from each of the model PoVs. The descriptors, image data, model PoVs, or other contextually relevant information are combined into key frame bundles having sufficient information to allow other computing devices to recognize the object at a later time.

First claim

What is claimed is:

1. A medical imaging system employing object ingestion comprising:
a non-transitory computer readable memory storing object ingestion software instructions; and
at least one processor coupled with the non-transitory computer readable memory and that is configurable upon execution of the object ingestion software instructions by the at least one processor to:
obtain a digital representation of a scene including an image of a target object captured by a sensor and a location;
identify contextually relevant shape objects relevant to a medical imaging context;
derive a set of features from the image of the target object;
select at least one target shape object from the contextually relevant shape objects based on the set of features;
generate a target object model from the at least one target shape object and portions of the image associated with the set of features;
create a set of key frame bundles from the target object model as a function of recognition algorithm descriptors and points of view associated with the at least one target shape object; and
send the set of key frame bundles to storage.

2. The system of claim 1 , wherein the image comprises at least one of the following types of image data: a still image, video data, a medical image, non-visible spectrum data, and an ultrasound image.

3. The system of claim 1 , wherein the contextually relevant shape objects have shape attributes that relate to the context.

4. The system of claim 3 , wherein the shape attributes comprise a symmetry attribute.

5. The system of claim 4 , wherein the symmetry attribute represents at least one of the following types of symmetry: rotational symmetry, scale symmetry, fractal symmetry, reflection symmetry, helical symmetry, and translation symmetry.

6. The system of claim 3 , wherein the shape attributes comprise at least one of the following attributes: a geometrical attribute, the location, a size, a distance, a width, a ratio, a thickness, a depth, a hole, a number of sides, a geometric center, a texture, a formula, a bounding box, a chirality, a periodicity, an orientation, an angular pitch, a scaling, a name, a key, a shape index, and a relevant descriptor.

7. The system of claim 1 , wherein the context comprises a positive association context.

8. The system of claim 7 , wherein the positive association context causes an increase in weight of the contextually relevant shape objects relative to other shape objects in selection of the at least one target shape object.

9. The system of claim 1 , wherein the context comprises a negative association context.

10. The system of claim 9 , wherein the negative association context causes a decrease in weight of other shape objects relative to the contextually relevant shape objects in selection of the at least one target shape object.

11. The medical imaging system of claim 1 , wherein the contextually relevant shape objects relevant to the medical imaging context comprise shape objects representing internal features of a body, including one or more of organs, bones, teeth, and the brain.

12. The medical imaging system of claim 1 , wherein a single heart shape model is used to generate object models from MRI imaging data, ultrasound data, X-ray data, or other forms of imaging data.

13. The medical imaging system of claim 3 , wherein a heart shape object model includes reference PoVs that are contextually relevant to a particular type of imaging modality, such as one of MRI or X-ray.

14. The system of claim 1 , wherein an object template represents at least one of the following types of objects: a tissue, and an organ.

15. The system of claim 1 , wherein the target object comprises at least one of the following: a building, a sign, a product, an automobile, a food, a document, a person, a face, clothing, a device, an organ, an animal, a plant, a game player, an inventory item, a book, a piece of laboratory equipment, a weapon, a landmark, a flower, an insect, and a plane.

16. The system of claim 1 , further comprising a mobile device operating as can object ingestion device and that includes the at least one processor and the non-transitory computer readable memory.

17. The system of claim 16 , wherein the mobile device comprises at least one of the following: a cell phone, a robot, a game console, a game interface, a digital camera, a medical device, a head-mount visor, a toy, and a vehicle.

18. The system of claim 1 , further comprising a shape database.

19. The system of claim 1 , further comprising an object recognition database.

20. The system of claim 1 , wherein the recognition algorithm descriptors include at least one of the following types of descriptors: a SIFT descriptor, a FREAK descriptor, a FAST descriptor, a SURF descriptor, a DAISY descriptor, and a BRISK descriptor.

21. The system of claim 1 , wherein the set of features comprises edges.

22. The system of claim 1 , further comprising determining a context associated with the scene based, at least in part, on the digital representation and the location.

23. A medical imaging method employing object ingestion comprising the steps of:
obtaining a digital representation of a scene including an image of a target object captured by a sensor and a location;
identifying contextually relevant shape objects relevant to a medical imaging context;
deriving a set of features from the image of the target object;
selecting at least one target shape object from the contextually relevant shape objects based on the set of features;
generating a target object model from the at least one target shape object and portions of the image associated with the set of features;
creating a set of key frame bundles from the target object model as a function of recognition algorithm descriptors and points of view associated with the at least one target shape object; and
sending the set of key frame bundles to storage.

24. The method of claim 23 , wherein the set of features comprises edges.

25. The method of claim 23 , further comprising determining a context associated with the scene based, at least in part, on the digital representation and the location.

26. A medical imaging device employing object ingestion comprising a mobile device that comprises at least one of a cell phone, a robot, a game console, a game interface, a digital camera, a medical device, a head-mount visor, a toy, and a vehicle, the mobile device comprising:
a non-transitory computer readable memory storing object ingestion software instructions; and
at least one processor coupled with the non-transitory computer readable memory and image sensor, and that is configurable upon execution of the object ingestion software instructions by the at least one processor to:
obtain a digital representation of a scene including an image of a target object captured by a sensor and a location;
identify contextually relevant shape objects relevant to a medical imaging context;
derive a set of features from the image of the target object;
select at least one target shape object from the contextually relevant shape objects based on the set of features;
generate a target object model from the at least one target shape object and portions of the image associated with the set of features;
create a set of key frame bundles from the target object model as a function of recognition algorithm descriptors and points of view associated with t

Assignees

• Nant Holdings Ip Llc

Inventors

• Wnuk Kamil
• Mckinnon David
• Sudol Jeremi
• Song Bing
• Siddiqui Matheen

Classifications

• G06V20/46 Primary

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06T2207/20061

  Hough transform · CPC title

• G06F16/5838

  using colour · CPC title

• G06T7/13

  Edge detection · CPC title