Generating synthetic images as training dataset for a machine learning network

Patent metadata

Field	Value
Publication number	US-11538171-B2
Application number	US-202117249378-A
Country	US
Kind code	B2
Filing date	Mar 1, 2021
Priority date	Apr 26, 2018
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

A method may include identifying a first image for training a deep learning network, wherein the first image includes at least one target object associated with at least one location in the first image, and wherein the first image is associated with a mask image; determining a set of deformations to create a training set of deformed images, wherein the training set is to be used to train the deep learning network; generating the training set of deformed images by applying the set of deformations to the first image; and generating a set of deformed mask images by applying the set of deformations to the mask image, wherein each deformed image of the training set of deformed images is associated with a respective mask image to identify the location of the at least one target object in each deformed image.

First claim

What is claimed is:

1. A method, comprising:
identifying, by a device, an image,
the image including one or more target objects;

generating, by the device, one or more deformed images by applying one or more deformations to the image;
generating, by the device, one or more deformed mask images by selectively applying the one or more deformations to one or more mask images associated with the image,
each deformed mask image, of the one or more deformed mask images, being associated with a deformed image, of the one or more deformed images, to identify at least one location corresponding to an object, of the one or more target objects, in the deformed image; and

training, by the device, a model using the one or more deformed images and the one or more deformed mask images.

2. The method of claim 1 , further comprising:
randomly or pseudorandomly selecting the one or more deformations; and
wherein generating the one or more deformed images comprises:
generating the one or more deformed images based on randomly or pseudorandomly selecting the one or more deformations.

3. The method of claim 1 , wherein generating the one or more deformed images by applying the one or more deformations to the image comprises:
generating the one or more deformed images by selectively applying a predetermined set of deformations to the image.

4. The method of claim 1 , further comprising:
determining the one or more deformations based on a type of the image.

5. The method of claim 4 , wherein determining the one or more deformations based on the type of the image comprises:
selecting the one or more deformations from a first set of deformations based on the type of the image being a first type; or
selecting the one or more deformations from a second set of deformations based on the type of the image being a second type.

6. The method of claim 1 , wherein the one or more deformations include at least one of:
one or more spatial deformations,
one or more lighting deformations,
one or more background deformations,
one or more atmospheric deformations,
one or more color deformations,
one or more deformations of one or more letters or numbers of the image,
an addition of the one or more target objects in the image,
a removal of the one or more target objects from the image,
an addition of one or more other target objects in the image, or
a relocation of the one or more target objects in the image.

7. The method of claim 1 , wherein generating the one or more deformed images by applying the one or more deformations to the image comprises:
generating a first deformed image, of the one or more deformed images, by applying a first deformation, of the one or more deformations, to the image; and
generating a second deformed image, of the one or more deformed images, by applying a second deformation, of the one or more deformations, to the first deformed image; and
wherein training the model using the one or more deformed images comprises:
training the model using the second deformed image.

8. A device, comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, configured to:
identify an image,
the image including one or more target objects;

generate one or more deformed images by applying one or more deformations to the image;
generate one or more deformed mask images by selectively applying the one or more deformations to one or more mask images associated with the image,
each deformed mask image, of the one or more deformed mask images, being associated with a deformed image, of the one or more deformed images, to identify at least one location corresponding to an object, of the one or more target objects, in the deformed image; and

train a model using the one or more deformed images and the one or more deformed mask images.

9. The device of claim 8 , wherein the one or more processors, when generating the one or more deformed images by applying the one or more deformations to the image, are configured to:
generate a particular deformed image, of the one or more deformed images, by selectively applying a deformation, of the one or more deformations, to the image after the image is rotated; and
wherein the one or more processors, when generating the one or more deformed mask images by selectively applying the one or more deformations to the one or more mask images, are configured to:
generate a particular deformed mask image, of the one or more deformed mask images, by selectively applying the deformation to a mask image, of the one or more mask images, corresponding to the particular deformed image.

10. The device of claim 8 , wherein the one or more processors, when generating the one or more deformed images by applying the one or more deformations to the image, are configured to:
generate a particular deformed image, of the one or more deformed images, by selectively applying a deformation, of the one or more deformations, to the image after the image is compressed; and
wherein the one or more processors, when generating the one or more deformed mask images by selectively applying the one or more deformations to the one or more mask images, are configured to:
generate a particular deformed mask image, of the one or more deformed mask images, by selectively applying the deformation to a mask image, of the one or more mask images, corresponding to the particular deformed image.

11. The device of claim 8 , wherein the one or more processors, when generating the one or more deformed mask images by selectively applying the one or more deformations to the one or more mask images, are configured to:
determine that a deformation, of the one or more deformations, is to be applied to a mask image, of the one or more mask images, based on a type of the deformation; and
selectively apply the deformation based on determining that the deformation is to be applied.

12. The device of claim 8 , wherein the one or more processors, when generating the one or more deformed mask images by selectively applying the one or more deformations to the one or more mask images, are configured to:
determine that a deformation, of the one or more deformations, has no effect on a mask image of the one or more mask images; and
determine that the deformation is not to be applied to the mask image based on determining that the deformation has no effect on the mask image.

13. The device of claim 8 , wherein the one or more processors are further configured to:
determine the one or more deformations based on a type of the image,
wherein the one or more deformations are selected from a first set of deformations based on the type of the image being a first type; or
wherein the one or more deformations are selected from a second set of deformations based on the type of the image being a second type.

14. The device of claim 8 , wherein the one or more processors, when generating the one or more deformed images by applying the one or more deformations to the image, are configured to:
generate a first deformed image, of the one or more deformed images, by applying a first deformation, of the one or more deformations, to the image; and
generate a second deformed image, of the one or more deformed images, by applying a second deformation, of the one or more deformations, to the first deformed image; and
wherein the one or more processors, when training the model using the one or more deformed images, are configured to:
train the model using the second deformed image.

15. A non-transitory computer-readable medium storing a set of instructions,

Assignees

• Capital One Services Llc

Inventors

• Farivar Reza
• Taylor Kenneth

Classifications

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/149 Primary

  involving deformable models, e.g. active contour models · CPC title

• G06T2207/30176

  Document · CPC title