Image generation using adversarial attacks for imbalanced datasets

Patent metadata

Field	Value
Publication number	US-12299082-B2
Application number	US-202418599029-A
Country	US
Kind code	B2
Filing date	Mar 7, 2024
Priority date	Jun 28, 2021
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

A method of balancing a dataset for a machine learning model includes identifying confusing classes of few-shot classes for a machine learning model during validation. One of the confusing classes and an image from one of the few-shot classes are selected. An image perturbation is computed such that the selected image is classified as the selected confusing class. The selected image is modified with the computed perturbation. The modified selected image is added to a batch for training the machine learning model.

First claim

What is claimed is:

1. A method of balancing a dataset for a machine learning model, the method comprising:
accessing a machine learning model implemented in a computing system;
identifying confusing classes of few-shot classes for the machine learning model during validation;
selecting one of the confusing classes;
selecting an image from one of the few-shot classes;
computing an image perturbation such that the selected image is classified as the selected confusing class, wherein the image perturbation is computed by applying a sequence of additive image perturbations;
modifying the selected image with the computed perturbation; and
adding the modified selected image to the one few-shot class for training the machine learning model.

2. The method of claim 1 , wherein the image perturbation is computed using a gradient-ascent technique that propagates a gradient to an input image.

3. The method of claim 2 , further comprising computing a pixel update based on the gradient.

4. The method of claim 1 , wherein the selected image is modified by maximizing a posterior probability or logit of a non-true class given an input image.

5. The method of claim 1 , wherein the one confusing class is selected by:
computing a probability distribution over all classes using confusion matrix scores for a tail class;
and using the computed probability distribution to sample for a confusing class.

6. The method of claim 1 , wherein a minimum class score is computed by randomly choosing a confidence value from within 0.15 and 0.25.

7. The method of claim 2 , wherein the gradient-ascent technique is executed with a learning rate δ=0.7.

8. The method of claim 7 , further comprising stopping the gradient-ascent technique when S c′ (I′)≥S c′ or when 15 iterations is reached.

9. A computing system, comprising:
one or more processors; and
a computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the computing system to perform operations comprising:
selecting a confusing class of few-shot classes for a machine learning model;
selecting an image from one of the few-shot classes;
computing an image perturbation such that the selected image is classified as the selected confusing class, wherein the image perturbation is computed using a gradient-ascent technique that propagates a gradient to an input image;
modifying the selected image with the computed perturbation; and
adding the modified selected image to a batch for training the machine learning model.

10. The computing system of claim 9 , wherein the image perturbation is computed using a gradient-ascent technique that propagates a gradient to an input image.

11. The computing system of claim 10 , further comprising computing a pixel update based on the gradient.

12. The computing system of claim 9 , wherein the selected image is modified by maximizing a posterior probability or logit of a non-true class given an input image.

13. The computing system of claim 9 , wherein the one confusing class is selected by:
computing a probability distribution over all classes using confusion matrix scores for a tail class;
and using the computed probability distribution to sample for a confusing class.

14. A computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising:
accessing a machine learning model implemented in a computing system;
identifying confusing classes of few-shot classes for the machine learning model during validation;
selecting one of the confusing classes;
selecting an image from one of the few-shot classes;
computing an image perturbation such that the selected image is classified as the selected confusing class, wherein the image perturbation is computed by applying a sequence of additive image perturbations;
modify the selected image with the computed perturbation; and
adding the modified selected image to a batch for training the machine learning model.

15. The computer-readable storage medium of claim 14 , wherein a minimum class score is computed by randomly choosing a confidence value from within 0.15 and 0.25.

16. The computer-readable storage medium of claim 15 , wherein the image perturbation is computed using a gradient-ascent technique that propagates a gradient to an input image.

17. The computer-readable storage medium of claim 16 , wherein:
the gradient-ascent technique is executed with a learning rate δ=0.7.

18. The computer-readable storage medium of claim 16 , further comprising stopping the gradient-ascent technique when S c′ (I′)≥S c′ or when 15 iterations is reached.

19. The computer-readable storage medium of claim 16 , further comprising computing a pixel update based on the gradient.

20. The computer-readable storage medium of claim 14 , wherein the selected image is modified by maximizing a posterior probability or logit of a non-true class given an input image.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Mittal Gaurav
• Karianakis Nikolaos
• Fragoso Rojas Victor Manuel
• Chen Mei
• Kozerawski Jedrzej Jakub

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title