Adversarial training method for noisy labels

Patent metadata

Field	Value
Publication number	US-11568324-B2
Application number	US-201916365485-A
Country	US
Kind code	B2
Filing date	Mar 26, 2019
Priority date	Dec 20, 2018
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

A system includes a memory; and a processor configured to train a first machine learning model based on the first dataset labeling; provide the second dataset to the trained first machine learning model to generate an updated second dataset including an updated second dataset labeling, determine a first difference between the updated second dataset labeling and the second dataset labeling; train a second machine learning model based on the updated second dataset labeling if the first difference is greater than a first threshold value; provide the first dataset to the trained second machine learning model to generate an updated first dataset including an updated first dataset labeling, determine a second difference between the updated first dataset labeling and the first dataset labeling; and train the first machine learning model based on the updated first dataset labeling if the second difference is greater than a second threshold value.

First claim

What is claimed is:

1. A system comprising:
a memory; and
a processor configured to execute instructions stored on the memory that, when executed by the processor, cause the processor to:
receive a first dataset comprising a first dataset labeling;
receive a second dataset comprising second dataset labeling;
train a first machine learning model based on the first dataset labeling;
provide the second dataset to the trained first machine learning model to generate an updated second dataset comprising an updated second dataset labeling, the updated second dataset being generated by classifying the second dataset using the trained first machine learning model;
determine a first difference between the updated second dataset labeling and the second dataset labeling;
train a second machine learning model based on the updated second dataset labeling if the first difference is greater than a first threshold value;
provide the first dataset to the trained second machine learning model to generate an updated first dataset comprising an updated first dataset labeling, the updated first dataset being generated by classifying the first dataset using the trained second machine learning model;
determine a second difference between the updated first dataset labeling and the first dataset labeling; and
train the first machine learning model based on the updated first dataset labeling if the second difference is greater than a second threshold value.

2. The system of claim 1 , wherein the instructions further cause the processor to continue to:
train the first machine learning model and the second machine learning model until the first difference is below the first threshold value and the second difference is below the second threshold value.

3. The system of claim 2 , wherein the first threshold value and the second threshold value are approximately equal to zero, wherein the first threshold value is different from the second threshold value.

4. The system of claim 1 , wherein the instructions further cause the processor to continue to train the first machine learning model and the second machine learning model until both the first machine learning model and the second machine learning model produce the same dataset labeling results when applied to the updated second dataset and the updated first dataset.

5. The system of claim 1 , wherein the updated first dataset is generated by updating data sample labels for the data from the first dataset that have flipped to a new class after the trained second machine learning model is applied to the first dataset.

6. The system of claim 1 , wherein the updated first dataset labeling is different from the first dataset labeling.

7. The system of claim 6 , wherein the first dataset is relabeled by live human inspectors according to the updated second dataset labeling to generate the updated first dataset.

8. The system of claim 1 , wherein the first machine learning model and the second machine learning model are classification algorithms.

9. The system of claim 1 , wherein the updated second dataset is generated by updating data sample labels for the data from the second dataset that have flipped to a new class after the trained first machine learning model is applied to the second dataset.

10. The system of claim 1 , wherein the first dataset is classified or labeled by live human inspectors.

11. A system comprising:
a memory; and
a processor configured to execute instructions stored on the memory that, when executed by the processor, cause the processor to:
train a first machine learning model based on a first dataset labeling of a first dataset;
provide a second dataset comprising second dataset labeling to the trained first machine learning model to generate an updated second dataset comprising an updated second dataset labeling;
train a second machine learning model based on the updated second dataset labeling;
provide the first dataset to the trained second machine learning model to generate an updated first dataset comprising an updated first dataset labeling; and
train the first machine learning model based on the updated first dataset labeling.

12. The system of claim 11 , wherein the instructions further cause the processor to continue to:
determine a first difference between the updated second dataset labeling and the second dataset labeling, wherein the second machine learning model is trained based on the updated second dataset labeling if the first difference is greater than a first threshold value;
determine a second difference between the updated first dataset labeling and the first dataset labeling, wherein the first machine learning model is trained based on the updated first dataset labeling if the second difference is greater than a second threshold value; and
train the first machine learning model and the second machine learning model until the first difference is below the first threshold value and the second difference is below the second threshold value.

13. The system of claim 12 , wherein the first threshold value and the second threshold value are approximately equal to zero, wherein the first threshold value is different from the second threshold value.

14. The system of claim 11 , wherein the updated first dataset being generated by classifying the first dataset using the trained second machine learning model and the updated second dataset is generated by classifying the second dataset using the trained first machine learning model.

15. The system of claim 11 , wherein the instructions further cause the processor to continue to train the first machine learning model and the second machine learning model until both the first machine learning model and the second machine learning model produce the same dataset labeling results when applied to the updated second dataset and the updated first dataset.

16. The system of claim 11 , wherein:
the updated first dataset is generated by updating data sample labels for the data from the first dataset that have flipped to a new class after the trained second machine learning model is applied to the first dataset,
the updated first dataset labeling is different from the first dataset labeling, and
the first dataset is relabeled by live human inspectors according to the updated second dataset labeling to generate the updated first dataset.

17. The system of claim 11 , wherein the first machine learning model and the second machine learning model are classification algorithms, wherein the first dataset is classified or labeled by live human inspectors, and wherein the updated second dataset is generated by updating data sample labels for the data from the second dataset that have flipped to a new class after the trained first machine learning model is applied to the second dataset.

18. A method comprising:
training, by a processor, a first machine learning model based on a first dataset labeling of a first dataset;
providing, by the processor, a second dataset comprising second dataset labeling to the trained first machine learning model to generate an updated second dataset comprising an updated second dataset labeling;
training, by the processor, a second machine learning model based on the updated second dataset labeling;
providing, by the processor, the first dataset to the trained second machine learning model to generate an updated first dataset comprising an updated first dataset labeling; and
training, by the processor, the first machine learning model based on the updated first dataset labeling.

19. The me

Assignees

• Samsung Display Co Ltd

Inventors

• Lee Janghwan

Classifications

• G06N20/20 Primary

  Ensemble learning · CPC title

• G06V10/7784 Primary

  based on feedback from supervisors · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F18/214 Primary

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title