Data augmentation for domain generalization

Patent metadata

Field	Value
Publication number	US-12277696-B2
Application number	US-202217716590-A
Country	US
Kind code	B2
Filing date	Apr 8, 2022
Priority date	Apr 8, 2022
Publication date	Apr 15, 2025
Grant date	Apr 15, 2025

Abstract

Methods and systems are disclosed for generating training data for a machine learning model for better performance of the model. A source image is selected from an image database, along with a target image. An image segmenter is utilized with the source image to generate a source image segmentation mask having a foreground region and a background region. The same is performed with the target image to generate a target image segmentation mask having a foreground region and a background region. Foregrounds and backgrounds of the source image and target image are determined based on the masks. The target image foreground is removed from the target image, and the source image foreground is inserted into the target image to create an augmented image having the source image foreground and the target image background. The training data for the machine learning model is updated to include this augmented image.

First claim

What is claimed is:

1. A system for performing at least one task with autonomous control of a part, the system comprising:
an image sensor configured to output an image of the part;
an actuator configured to bin the part based on a detected defect in the part;
a processor; and
memory including instructions that, when executed by the processor, cause the processor to:
utilize an image segmenter on a source image stored in the memory to generate a source image segmentation mask having a foreground region and a background region;
utilize the image segmenter on a target image stored in the memory to generate a target image segmentation mask having a foreground region and a background region;
determine a source image foreground and a source image background of the source image based on the source image segmentation mask;
determine a target image foreground and a target image background of the target image based on the target image segmentation mask;
remove the target image foreground from the target image;
insert the source image foreground into the target image with the removed target image foreground to create an augmented image having the source image foreground and the target image background;
update training data of the machine learning model with the augmented image;
utilize the machine learning model with the updated training data to determine a defect in the part; and
actuate the actuator to bin the part based on the determined defect in the part.

2. The system of claim 1 , wherein the instructions, when executed by the processor, cause the processor to remove the target image foreground from the target image background by:
determining an average color of pixels in the target image background; and
changing the pixels in the target image foreground to assume the average color.

3. A computer-implemented method for generating training data for a machine learning model, the computer-implemented method comprising:
selecting a source image from an image database;
selecting a target image from the image database;
utilizing an image segmenter with the source image to generate a source image segmentation mask having a foreground region and a background region;
utilizing the image segmenter with the target image to generate a target image segmentation mask having a foreground region and a background region;
determining a source image foreground and a source image background of the source image based on the source image segmentation mask;
determining a target image foreground and a target image background of the target image based on the target image segmentation mask;
removing the target image foreground from the target image;
inserting the source image foreground into the target image with the removed target image foreground to create an augmented image having the source image foreground and the target image background; and
updating training data of the machine learning model with the augmented image.

4. The computer-implemented method of claim 3 , wherein the step of removing includes:
changing colors of at least a group of pixels in the target image foreground.

5. The computer-implemented method of claim 3 , wherein the step of removing includes:
determining an average color of pixels in the target image background; and
changing the pixels in the target image foreground to assume the average color.

6. The computer-implemented method of claim 3 , further comprising:
determining a color of a pixels in the source image at a location corresponding to the source image foreground; and
wherein the step of inserting includes assigning the color to pixels in the target image at a location corresponding to the target image foreground.

7. The computer-implemented method of claim 3 , wherein:
the source image segmentation mask and the target image segmentation mask are generated with a first color in the respective foreground regions and a second color in the respective background regions.

8. The computer-implemented method of claim 3 , further comprising outputting a trained machine learning model based on the updated training data.

9. The computer-implemented method of claim 8 , further comprising:
receiving an input image of a part from an image sensor;
using the trained machine learning model and the input image to determine a defect is present in the part; and
binning the part based on the determined defect in the part.

10. The computer-implemented method of claim 3 , wherein the source image is part of a group of source images having a common domain, and wherein the step of selecting the source image is based on a probability such that the probability of selecting a given source image decreases as the number of images with the common domain increases.

11. The computer-implemented method of claim 3 , wherein the image segmenter is an image segmentation machine learning model.

12. A system for training a machine learning model, the system comprising:
a computer-readable storage medium configured to store computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions, the computer-executable instructions comprising:
utilizing an image segmenter with a source image to generate a source image segmentation mask having a foreground region and a background region;
utilizing the image segmenter with a target image to generate a target image segmentation mask having a foreground region and a background region;
determining a source image foreground and a source image background of the source image based on the source image segmentation mask;
determining a target image foreground and a target image background of the target image based on the target image segmentation mask;
removing the target image foreground from the target image;
inserting the source image foreground into the target image with the removed target image foreground to create an augmented image having the source image foreground and the target image background; and
updating training data of the machine learning model with the augmented image.

13. The system of claim 12 , wherein the step of removing includes:
changing colors of at least a group of pixels in the target image foreground.

14. The system of claim 12 , wherein the step of removing includes:
determining an average color of pixels in the target image background; and
changing the pixels in the target image foreground to assume the average color.

15. The system of claim 12 , wherein the computer-executable instructions further comprise:
determining a color of a pixels in the source image at a location corresponding to the source image foreground; and
wherein the step of inserting includes assigning the color to pixels in the target image at a location corresponding to the target image foreground.

16. The system of claim 12 , wherein:
the source image segmentation mask and the target image segmentation mask are generated with a first color in the respective foreground regions and a second color in the respective background regions.

17. The system of claim 12 , wherein the computer-executable instructions further comprise:
outputting a trained machine learning model based on the updated training data.

18. The system of claim 17 , wherein the computer-executable instructions further comprise:
receiving an input image of a part from an image source;
using the trained machine learning model and the input image to determine a defect is present in the part; and
outputting a message indicating the presence of the defect

Assignees

• Bosch Gmbh Robert

Inventors

• Beggel Laura
• Cabrita Condessa Filipe J
• Hutmacher Robin
• Kolter Jeremy
• Ngo Nhung Thi Phuong
• Sheikholeslami Fatemeh
• Willmott Devin T

Classifications

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06N20/00

  Machine learning · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title