Evaluation method, non-transitory computer-readable storage medium, and information processing device

Patent metadata

Field	Value
Publication number	US-2022277174-A1
Application number	US-202217750641-A
Country	US
Kind code	A1
Filing date	May 23, 2022
Priority date	Dec 4, 2019
Publication date	Sep 1, 2022
Grant date	—

Abstract

An evaluation method performed by a computer, the evaluation method includes generating a plurality of subsets that contain one or more pieces of training data, based on a set of a plurality of pieces of training data that includes pairs of input data and labels for machine learning, generating a trained model configured to estimate the labels from the input data, for each of the subsets, by performing the machine learning that uses the training data contained in the subsets, and performing evaluation related to aggression to the machine learning in the training data contained in the subsets, for each of the subsets, based on estimation accuracy of the trained model generated by using the training data contained in the subsets.

First claim

What is claimed is:

1 . An evaluation method performed by a computer, the evaluation method comprising:
generating a plurality of subsets that contain one or more pieces of training data, based on a set of a plurality of pieces of training data that includes pairs of input data and labels for machine learning; generating a trained model configured to estimate the labels from the input data, for each of the subsets, by performing the machine learning that uses the training data contained in the subsets; and performing evaluation related to aggression to the machine learning in the training data contained in the subsets, for each of the subsets, based on estimation accuracy of the trained model generated by using the training data contained in the subsets.

2 . The evaluation method according to claim 1 , wherein
the evaluation includes evaluating the aggression to the machine learning in the training data contained in the subsets higher as the estimation accuracy of the trained models generated based on the subsets is lower.

3 . The evaluation method according to claim 1 , wherein
the generating the subsets, the generating the trained models, and the evaluation are repeated based on the set of a predetermined number of pieces of the training data contained in the subsets from one with the highest aggression indicated by the evaluation.

4 . The evaluation method according to claim 1 , wherein
the generating the subsets includes performing clustering in which the training data is classified into one of a plurality of clusters, based on similarity between the training data, and for the training data classified into a predetermined number of the respective clusters from one with a smallest number of pieces of the belonging training data, including particular pieces of the training data that belong to a same cluster into a common one of the subsets.

5 . The evaluation method according to claim 1 , wherein
the generating the subsets, the generating the trained models, and the evaluation are repeated, and each time the evaluation is performed, contamination candidate points are added to a predetermined number of pieces of the training data contained in the subsets from one with the highest aggression indicated by the evaluation, and the predetermined number of pieces of the training data from one with the highest contamination candidate points are output.

6 . A non-transitory computer-readable storage medium storing an evaluation program that causes a processor included in a noise estimation apparatus to execute a process, the process comprising:
generating a plurality of subsets that contain one or more pieces of training data, based on a set of a plurality of pieces of training data that includes pairs of input data and labels for machine learning; generating a trained model configured to estimate the labels from the input data, for each of the subsets, by performing the machine learning that uses the training data contained in the subsets; and performing evaluation related to aggression to the machine learning in the training data contained in the subsets, for each of the subsets, based on estimation accuracy of the trained model generated by using the training data contained in the subsets.

7 . An information processing device comprising:
a memory; and a processor coupled to the memory and configured to: generate a plurality of subsets that contain one or more pieces of training data, based on a set of a plurality of pieces of training data that includes pairs of input data and labels for machine learning, generate a trained model configured to estimate the labels from the input data, for each of the subsets, by performing the machine learning that uses the training data contained in the subsets, and perform evaluation related to aggression to the machine learning in the training data contained in the subsets, for each of the subsets, based on estimation accuracy of the trained model generated by using the training data contained in the subsets.

Assignees

• Fujitsu Ltd

Inventors

• Shimizu Toshiya

Classifications

• G06F18/217 Primary

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title