System, method and computer program product for training a three dimensional object indentification system and identifying three dimensional objects using semantic segments

Patent metadata

Field	Value
Publication number	US-2017193340-A1
Application number	US-201514983834-A
Country	US
Kind code	A1
Filing date	Dec 30, 2015
Priority date	Dec 30, 2015
Publication date	Jul 6, 2017
Grant date	—

Abstract

A method of training an object identification system and identifying three dimensional objects using semantic segments includes receiving, into a non-volatile memory, an input file containing a geometric description of a three dimensional object having one or more semantic segments and one or more annotations for each of the one or more semantic segments, receiving, into the non-volatile memory one or more training images of the three dimensional object, identifying, through a processor, the one or more segments in the one or more training images, computing, through a training module, one or more descriptors to the one or more segments, and generating an output file representing a machine vision of the three dimensional object.

First claim

1 . A method of training a three dimensional object identification system and identifying three dimensional objects using semantic segments comprising:
receiving, into a non-volatile memory, an input file containing a geometric shape description of a three dimensional object having one or more semantic segments and one or more annotations for each of the one or more semantic segments; receiving, into the non-volatile memory one or more training images of the object; identifying, through a processor, the one or more semantic segments in the one or more training images; computing, through a training module, one or more descriptors for the one or more identified semantic segments; and generating an output file representing a machine vision of the three dimensional object.

2 . The method of claim 1 , wherein computing the one or more descriptors include determining at least one of a color, a texture, material characteristics, and an area of interest to each of the one or more semantic segments.

3 . The method of claim 1 , further comprising: identifying stability of each semantic segment by analyzing a distribution of descriptors in each semantic segment.

4 . The method of claim 3 , wherein identifying stability of each semantic segment includes analyzing a distribution of the one or more descriptors in each of the one or more training images.

5 . The method of claim 4 , wherein analyzing the distribution of the one or more descriptors includes identifying whether a semantic segment includes one of a single stable appearance, multiple stable appearances, and no stable appearances.

6 . The method of claim 5 , wherein identifying whether the segment includes a single stable appearance includes determining a unimodal concentration distribution of descriptors meets a predetermined criteria.

7 . The method of claim 5 , wherein identifying whether the segment includes multiple stable appearances includes determining a multimodal concentration distribution of descriptors meets a predetermined criteria.

8 . The method of claim 5 , wherein identifying whether the segment includes no stable appearances includes determining a multimodal concentration distribution of descriptors fails to meet a predetermined criteria.

9 . The method of claim 1 , further comprising: identifying a three dimensional object with the output file including receiving one or more query images of the three dimensional object, identifying one or more semantic segments in the one or more query images based on the one or more descriptors in the output file, identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments, and generating an object identified signal.

10 . The method of claim 9 , wherein identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments includes identifying a semantic segment in the one or more query images that does not appear in the one or more training images.

11 . A system comprising:
a central processor unit (CPU); a non-volatile memory operatively connected to the CPU; and a training module configured to analyze an input file containing a geometric shape description of a three dimensional object having one or more semantic segments, and one or more annotations for each of the one or more semantic segments, the training module including computer readable program code embodied therewith, the computer readable program code, when executed by the CPU, causes the CPU to:
receive, into a non-volatile memory, an input file containing the geometric shape description of the three-dimensional object having the one or more semantic segments and the one or more annotations for each of the one or more semantic segments;
receive, into the non-volatile memory one or more training images of the three dimensional object;
identify, through a processor, the one or more segments in the one or more training images;
compute, through the training module, one or more descriptors to the identified one or more segments; and
generate an output file representing a machine vision of the three dimensional object.

12 . The system of claim 11 , wherein the computer readable program code, when executed by the processor, causes the processor to: determine at least one of a color, a texture, material characteristics, and an area of interest to each of the one or more semantic segments when computing the one or more descriptors.

13 . The system of claim 11 , wherein the computer readable program code, when executed by the processor, causes the processor to: identify stability of each semantic segment by analyzing a distribution of descriptors in each semantic segment.

14 . The system of claim 13 , wherein the computer readable program code, when executed by the processor, causes the processor to: analyze a distribution of the one or more descriptors in each of the one or more training images when identifying stability of each semantic segment.

15 . The system of claim 11 , wherein the computer readable program code, when executed by the processor, causes the processor to: identify a three dimensional object with the output file including receiving one or more query images of the three dimensional object, identifying one or more semantic segments in the one or more query images based on the one or more descriptors in the output file, identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments, and generating an object identified signal.

16 . The system of claim 15 , wherein the computer readable program code, when executed by the processor, causes the processor to: identify a portion of a semantic segment in the one or more query images that does not appear in the one or more training images when identifying one or more objects in the captured images as a spatial configuration of the one or more semantic segments.

17 . A computer program product for analyzing an input file containing a geometric shape description of a three dimensional object having one or more semantic segments, and one or more annotations for the one or more semantic segments, the computer program product comprising a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, when executed by a processor, causing the processor to:
receive, into a non-volatile memory, the input file containing the geometric shape description of the three dimensional object having the one or more semantic segments and the one or more annotations for each of the one or more semantic segments; receive, into the non-volatile memory one or more training images of the three dimensional object; identify, through a processor, the one or more segments in the one or more training images; compute, through a training module, one or more descriptors to the identified one or more segments; and generate an output file representing a machine vision of the three dimensional object.

18 . The computer program product of claim 17 , wherein the computer readable program code, when executed by the processor, causes the processor to: determine at least one of a color, a texture, material characteristics, and an area of interest to each of the one or more semantic segments when computing the one or more descriptors.

19 . The computer program product of claim 17 , wherein the computer readable program code, when executed by the processor, causes the processor to:
identify stabili

Assignees

• IBM

Inventors

• Shtok Joseph
• Tzadok Asaf
• Tzur Yochay

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06V30/19147

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V30/274

  Syntactic or semantic context, e.g. balancing · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T2207/20061

  Hough transform · CPC title