Generation of synthetic image data

Patent metadata

Field	Value
Publication number	US-11568576-B1
Application number	US-202017117784-A
Country	US
Kind code	B1
Filing date	Dec 10, 2020
Priority date	Dec 10, 2020
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

Techniques are generally described for generation of photorealistic synthetic image data. A generator network generates first synthetic image data. A first class of image data represented by a first portion of the first synthetic image data is detected and the first portion is sent to a first discriminator network. The first discriminator network generates a prediction of whether the first portion of the first synthetic image data is synthetically generated. A second class of image data represented by a second portion of the first synthetic image data is detected and the second portion is sent to a second discriminator network. The second discriminator network generates a prediction of whether the second portion of the first synthetic image data is synthetically generated. The generator network is updated based on the predictions of the discriminators.

First claim

What is claimed is:

1. A method of generating synthetic images, comprising:
generating, by a generator network, first synthetic image data;
detecting a first class of image data represented by a first portion of the first synthetic image data;
sending the first portion of the first synthetic image data to a first discriminator network, wherein the first discriminator network is associated with the first class of image data;
generating, by the first discriminator network, first data comprising a first prediction of whether the first portion of the first synthetic image data is synthetically generated;
detecting a second class of image data represented by a second portion of the first synthetic image data;
sending the second portion of the first synthetic image data to a second discriminator network, wherein the second discriminator network is associated with the second class of image data;
generating, by the second discriminator network, second data comprising a second prediction of whether the second portion of the first synthetic image data is synthetically generated;
updating at least one parameter of the generator network based at least in part on the first data and the second data; and
generating, by the generator network, second synthetic image data.

2. The method of claim 1 , further comprising:
sending the first synthetic image data to a first machine learning algorithm;
detecting, by the first machine learning algorithm, the first class of image data represented by the first portion of the first synthetic image data;
detecting, by the first machine learning algorithm, the second class of image data represented by the second portion of the first synthetic image data;
generating first cropped image data comprising the first portion; and
generating second cropped image data comprising the second portion.

3. The method of claim 1 , further comprising:
determining that the first discriminator network corresponds to the first class of a first object, wherein the first portion of the first synthetic image data is sent to the first discriminator network based on the determination that the first discriminator network corresponds to the first class of the first object; and
determining that the second discriminator network corresponds to the second class of a second object, wherein the second portion of the first synthetic image data is sent to the second discriminator network based on the determination that the second discriminator network corresponds to the second class of the second object.

4. The method of claim 1 , further comprising sending the first synthetic image data to a third discriminator network, wherein the third discriminator network is trained to predict whether image data representing an ensemble of objects is a real image or a synthetically-generated image.

5. The method of claim 1 , further comprising:
generating a first vector representation of the first portion;
determining a second vector representation in a multi-dimensional vector space based on a distance between the first vector representation and the second vector representation, wherein the second vector representation corresponds to an item available for purchase via an e-commerce service; and
displaying a listing for the item on a display.

6. The method of claim 1 , further comprising generating, by the generator network, the second synthetic image data, wherein the first synthetic image data and the second synthetic image data are respective frames of a video.

7. The method of claim 6 , further comprising determining an action that a first object represented by the first portion is undertaking based at least in part on the first synthetic image data and the second synthetic image data.

8. The method of claim 7 , further comprising:
determining a third discriminator network corresponding to the action; and
sending the first synthetic image data and the second synthetic image data to the third discriminator network.

9. The method of claim 1 , further comprising:
receiving a first cropped image of the first portion, wherein the first cropped image was generated using a camera;
generating a first label for the first cropped image of the first portion indicating that the first cropped image is a real image of a first object;
receiving a second cropped image of the first object, wherein the second cropped image was synthetically generated;
generating a second label for the second cropped image of the first object indicating that the second cropped image is a synthetically-generated image of the first object; and
updating at least one parameter of the first discriminator network using the first cropped image, the second cropped image, the first label, and the second label.

10. A system comprising:
at least one processor; and
non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to:
generate, by a generator network, first synthetic image data;
detect a first class of image data represented by a first portion of the first synthetic image data;
send the first portion of the first synthetic image data to a first discriminator network, wherein the first discriminator network is associated with the first class of image data;
generate, by the first discriminator network, first data comprising a first prediction of whether the first portion of the first synthetic image data is synthetically generated;
detect a second class of image data represented by a second portion of the first synthetic image data;
send the second portion of the first synthetic image data to a second discriminator network, wherein the second discriminator network is associated with the second class of image data;
generate, by the second discriminator network, second data comprising a second prediction of whether the second portion of the first synthetic image data is synthetically generated;
update at least one parameter of the generator network based at least in part on the first data or the second data; and
generate, by the generator network, second synthetic image data.

11. The system of claim 10 , the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor are further effective to:
send the first synthetic image data to a convolutional neural network (CNN);
detect, by the CNN, the first class of image data represented by the first portion of the first synthetic image data;
detect, by the CNN, the second class of image data represented by the second portion of the first synthetic image data;
generate first cropped image data comprising the first portion; and
generate second cropped image data comprising the second portion.

12. The system of claim 10 , the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor are further effective to:
determine that the first discriminator network corresponds to the first class of a first object, wherein the first portion of the first synthetic image data is sent to the first discriminator network based on the determination that the first discriminator network corresponds to the first class of the first object; and
determine that the second discriminator network corresponds to the second class of a second object, wherein the second portion of the first synthetic image data is sent to the second discriminator network based on the determination that the second discriminator network corresponds to the second class of the second object.

13. The system of claim 10 , the non-transitory computer-readable memory storing further instru

Assignees

• Amazon Tech Inc

Inventors

• Martinez Aleix Margarit
• Gadde Raghu Deep
• Feng Qianli
• Indrei Alexandru
• Gjonej Gerard

Classifications

• G06K9/6217

  Physics · mapped topic

• G06K9/6267

  Physics · mapped topic

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/20132

  Image cropping · CPC title