Systems and methods for automatically generating training image sets for an object

Patent metadata

Field	Value
Publication number	US-10885386-B1
Application number	US-201916571533-A
Country	US
Kind code	B1
Filing date	Sep 16, 2019
Priority date	Sep 16, 2019
Publication date	Jan 5, 2021
Grant date	Jan 5, 2021

Abstract

A computer-implemented method for generating a training image set includes retrieving, from at least one memory device, model data corresponding to a three-dimensional (3-D) model of a target object, and creating a plurality of two-dimensional (2-D) synthetic images from the model data. The 2-D synthetic images include a plurality of views of the 3-D model. The method also includes creating a plurality of semantic segmentation images by identifying a plurality of pixels that define the target object in the 2-D synthetic image, and assigning a semantic segmentation label to the identified pixels of the target object. The method further includes generating linking data associating each of the semantic segmentation labels with a corresponding one of the 2-D synthetic images, and storing the training image set including the 2-D synthetic images, the semantic segmentation labels, and the linking data.

First claim

What is claimed is:

1. A method for generating a training image set, the method implemented on a computing system comprising at least one processor in communication with at least one memory device, the method comprising using the at least one processor to:
retrieve, from the at least one memory device, model data corresponding to a three-dimensional (3-D) model of a target object;
create a plurality of two-dimensional (2-D) synthetic images from the model data, the 2-D synthetic images including a plurality of views of the 3-D model;
create a plurality of semantic segmentation images by:
identifying a plurality of pixels that define the target object in the 2-D synthetic images; and
assigning a semantic segmentation label to the identified pixels of the target object;

generate linking data associating each of the semantic segmentation images with a corresponding one of the 2-D synthetic images; and
store the training image set including the 2-D synthetic images, the semantic segmentation images, and the linking data.

2. The method according to claim 1 , wherein the plurality of views includes a plurality of angular perspectives, and wherein creating the plurality of 2-D synthetic images comprises using the at least one processor to:
rotate the 3-D model through the plurality of angular perspectives; and
create one of the 2-D synthetic images at each of the plurality of angular perspectives.

3. The method according to claim 2 , wherein the plurality of views further includes a plurality of distance perspectives, the method further comprising using the at least one processor to create the plurality of 2-D synthetic images by:
zooming the 3-D model through the plurality of distance perspectives; and
creating one of the 2-D synthetic images at each of the plurality of angular perspectives for each of the distance perspectives.

4. The method according to claim 1 , wherein the target object is presentable in a plurality of configurations and the model data includes configuration data representative of each of the configurations, and wherein using the at least one processor to create the plurality of 2-D synthetic images includes:
selectively applying, to the 3-D model, the configuration data for each of the configurations; and
creating the plurality of 2-D synthetic images at each of the plurality of views for each of the configurations.

5. The method according to claim 1 , further comprising using the at least one processor to create the plurality of 2-D synthetic images includes:
creating a plurality of 2-D synthetic base images each corresponding to one of the views of the 3-D model; and
superimposing the 2-D synthetic base images over a plurality of different backgrounds.

6. The method according to claim 1 , further comprising:
retrieving, from the at least one memory device, occlusion image data corresponding to an occluding object, wherein creating the plurality of 2-D synthetic images includes:
creating a plurality of 2-D synthetic base images each corresponding to one of the views of the 3-D model; and
superimposing the occlusion image data over each of the 2-D synthetic base images.

7. The method according to claim 1 , wherein using the at least one processor to create the plurality of 2-D synthetic images includes:
creating a plurality of 2-D synthetic base images each corresponding to one of the views of the 3-D model; and
applying at least one of extrinsic sensor effects and intrinsic sensor effects to the 2-D synthetic base images.

8. The method according to claim 1 , further comprising using the at least one processor to, for at least one of the 2-D synthetic images:
create an additional label including at least one of:
a perspective label corresponding to a perspective of the corresponding one of the views of the 3-D model used to create the at least one 2-D synthetic image;
a background label corresponding to one of a plurality of backgrounds used to create the at least one 2-D synthetic image; and
an occlusion label corresponding to an occluding object for which occlusion image data is included in the at least one 2-D synthetic image;

generate additional linking data associating the additional label with the corresponding at least one 2-D synthetic image; and
store the additional linking data as a portion of the training image set.

9. The method according to claim 8 , wherein the additional linking data comprises an inclusion of the additional label in at least one of a file name and a file header of the at least one 2-D synthetic image.

10. The method according to claim 1 , further comprising using the at least one processor to generate the linking data by including at least a portion of a file name of the corresponding 2-D synthetic image in at least one of a file name and a file header of a file containing the semantic segmentation image.

11. A computing system for generating a training image set, the computing system comprising at least one processor in communication with at least one memory device, wherein the at least one processor is configured to:
retrieve, from the at least one memory device, model data corresponding to a three-dimensional (3-D) model of a target object;
create a plurality of two-dimensional (2-D) synthetic images from the model data, the 2-D synthetic images including a plurality of views of the 3-D model;
create a plurality of semantic segmentation images by:
identifying a plurality of pixels that define the target object in the 2-D synthetic images; and
assigning a semantic segmentation label to the identified pixels of the target object;

generate linking data associating each of the semantic segmentation images with a corresponding one of the 2-D synthetic images; and
store the training image set including the 2-D synthetic images, the semantic segmentation images, and the linking data.

12. The computing system according to claim 11 , wherein the plurality of views includes a plurality of angular perspectives, and wherein the at least one processor is further configured to create the plurality of 2-D synthetic images by:
rotating the 3-D model through the plurality of angular perspectives; and
creating one of the 2-D synthetic images at each of the plurality of angular perspectives.

13. The computing system according to claim 12 , wherein the plurality of views further includes a plurality of distance perspectives, and wherein the at least one processor is further configured to create the plurality of 2-D synthetic images by:
zooming the 3-D model through the plurality of distance perspectives; and
creating one of the 2-D synthetic images at each of the plurality of angular perspectives for each of the distance perspectives.

14. The computing system according to claim 11 , wherein the target object is presentable in a plurality of configurations and the model data includes configuration data representative of each of the configurations, and wherein the at least one processor is further configured to create the plurality of 2-D synthetic images at least in part by:
selectively applying, to the 3-D model, the configuration data for each of the configurations; and
creating the plurality of 2-D synthetic images at each of the plurality of views for each of the configurations.

15. The computing system according to claim 11 , wherein the at least one processor is further configured to create the plurality of 2-D synthetic images at least in part by:
creating a plurality of 2-D synthetic base images each corresponding to one of the views of the 3-D model; and
superimposing the 2-D synthetic base images over a plurality

Assignees

• Boeing Co

Inventors

• Evans Nick Shadbeh
• Muir Eric Raymond
• Staudinger Tyler C

Classifications

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06F18/214 Primary

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F18/24

  Classification techniques · CPC title