System, method and computer program product for training a three dimensional object indentification system and identifying three dimensional objects using semantic segments

Patent metadata

Field	Value
Publication number	US-9836673-B2
Application number	US-201514983834-A
Country	US
Kind code	B2
Filing date	Dec 30, 2015
Priority date	Dec 30, 2015
Publication date	Dec 5, 2017
Grant date	Dec 5, 2017

Abstract

A method of training an object identification system and identifying three dimensional objects using semantic segments includes receiving, into a non-volatile memory, an input file containing a geometric description of a three dimensional object having one or more semantic segments and one or more annotations for each of the one or more semantic segments, receiving, into the non-volatile memory one or more training images of the three dimensional object, identifying, through a processor, the one or more segments in the one or more training images, computing, through a training module, one or more descriptors to the one or more segments, and generating an output file representing a machine vision of the three dimensional object.

First claim

What is claimed is:

1. A method of training a three dimensional object identification system and identifying three dimensional objects using semantic segments comprising:
receiving, into a non-volatile memory, an input file containing a geometric shape description of a three dimensional object having one or more semantic segments and one or more annotations for each of the one or more semantic segments;
receiving, into the non-volatile memory one or more training images of the object;
identifying, through a processor, the one or more semantic segments in the one or more training images;
computing, through a training module, one or more descriptors for the one or more identified semantic segments by determining at least one of a color and a texture of the one or more semantic segments; and
generating an output file representing a machine vision of the three dimensional object based on the one or more descriptors for the one or more identified semantic segments computed through the training module.

2. The method of claim 1 , wherein computing the one or more descriptors includes determining an area of interest of the one or more semantic segments.

3. The method of claim 1 , further comprising: identifying stability of each semantic segment by analyzing a distribution of descriptors in each semantic segment.

4. The method of claim 3 , wherein identifying stability of each semantic segment includes analyzing a distribution of the one or more descriptors in each of the one or more training images.

5. The method of claim 4 , wherein analyzing the distribution of the one or more descriptors includes identifying whether a semantic segment includes one of a single stable appearance, multiple stable appearances, and no stable appearances.

6. The method of claim 5 , wherein identifying whether the segment includes a single stable appearance includes determining a unimodal concentration distribution of descriptors meets a predetermined criteria.

7. The method of claim 5 , wherein identifying whether the segment includes multiple stable appearances includes determining a multimodal concentration distribution of descriptors meets a predetermined criteria.

8. The method of claim 5 , wherein identifying whether the segment includes no stable appearances includes determining a multimodal concentration distribution of descriptors fails to meet a predetermined criteria.

9. The method of claim 1 , further comprising: identifying a three dimensional object with the output file including receiving one or more query images of the three dimensional object, identifying one or more semantic segments in the one or more query images based on the one or more descriptors in the output file, identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments, and generating an object identified signal.

10. The method of claim 9 , wherein identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments includes identifying a semantic segment in the one or more query images that does not appear in the one or more training images.

11. The method of claim 1 , further comprising:
detecting an area of instability including a change in material identified in the training image; and
determining if the area of instability is a known area of instability.

12. A system comprising:
a central processor unit (CPU);
a non-volatile memory operatively connected to the CPU; and
a training module configured to analyze an input file containing a geometric shape description of a three dimensional object having one or more semantic segments, and one or more annotations for each of the one or more semantic segments, the training module including computer readable program code embodied therewith, the computer readable program code, when executed by the CPU, causes the CPU to:
receive, into a non-volatile memory, an input file containing the geometric shape description of the three-dimensional object having the one or more semantic segments and the one or more annotations for each of the one or more semantic segments;
receive, into the non-volatile memory one or more training images of the three dimensional object;
identify, through a processor, the one or more segments in the one or more training images;
compute, through the training module, one or more descriptors to the identified one or more segments by determining at least one of a color and a texture of the one or more semantic segments; and
generate an output file representing a machine vision of the three dimensional object based on the one or more descriptors for the one or more identified semantic segments computed through the training module.

13. The system of claim 12 , wherein the computer readable program code, when executed by the CPU, causes the CPU to: determine an area of interest of the one or more semantic segments when computing the one or more descriptors.

14. The system of claim 12 , wherein the computer readable program code, when executed by the CPU, causes the CPU to: identify stability of each semantic segment by analyzing a distribution of descriptors in each semantic segment.

15. The system of claim 14 , wherein the computer readable program code, when executed by the CPU, causes the CPU to: analyze a distribution of the one or more descriptors in each of the one or more training images when identifying stability of each semantic segment.

16. The system of claim 12 , wherein the computer readable program code, when executed by the CPU, causes the CPU to: identify a three dimensional object with the output file including receiving one or more query images of the three dimensional object, identifying one or more semantic segments in the one or more query images based on the one or more descriptors in the output file, identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments, and generating an object identified signal.

17. The system of claim 16 , wherein the computer readable program code, when executed by the CPU, causes the CPU to: identify a portion of a semantic segment in the one or more query images that does not appear in the one or more training images when identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments.

18. The system of claim 12 , wherein the computer readable program code, when executed by the CPU, causes the CPU to:
detect an area of instability including a change in material identified in the training image; and
determine if the area of instability is a known area of instability.

19. A computer program product for analyzing an input file containing a geometric shape description of a three dimensional object having one or more semantic segments, and one or more annotations for the one or more semantic segments, the computer program product comprising a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, when executed by a processor, causing the processor to:
receive, into a non-volatile memory, the input file containing the geometric shape description of the three dimensional object having the one or more semantic segments and the one or more annotations for each of the one or more semantic segments;
receive, into the non-volatile memory one or more training images of the three dimensional object;
identify, through a processor, the one or more segments in the one or mor

Assignees

• IBM

Inventors

• Shtok Joseph
• Tzadok Asaf
• Tzur Yochay

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06V30/19147

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V30/274

  Syntactic or semantic context, e.g. balancing · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T2207/20061

  Hough transform · CPC title