Learning device estimating apparatus, learning device estimating method, risk evaluation apparatus, risk evaluation method, and program

Patent metadata

Field	Value
Publication number	US-11847230-B2
Application number	US-201917281985-A
Country	US
Kind code	B2
Filing date	Sep 18, 2019
Priority date	Oct 10, 2018
Publication date	Dec 19, 2023
Grant date	Dec 19, 2023

Abstract

A learning device estimating apparatus aims at a learning device as an attack target, and comprises a recording part, an inquiring part, a capturing part and a learning part. A predetermined plurality of pieces of observation data are recorded. The inquiring part inquires of the attack target learning device for each of the pieces of observation data recorded in the recording part to acquire label data and records the acquired label data to the recording part in association with observation data. The capturing part inputs the observation data and the label data associated with the observation data that have been recorded to the recording part, to the learning part. The learning part is characterized by using an activation function that outputs a predetermined ambiguous value in a process for determining a classification prediction result, and the learning part performs learning using the inputted observation data and label data.

First claim

What is claimed is:

1. A learning device estimating apparatus for use with a learning device, for classification task that outputs a type of inputted observation data as label data, as an attack target, the learning device estimating apparatus comprising:
a memory;
inquiring circuitry;
capturing circuitry; and
learning circuitry,
wherein:
the memory stores a predetermined plurality of pieces of observation data,
the inquiring circuitry inquires of the attack target learning device for each of the pieces of observation data recorded in the memory to acquire label data and records the acquired label data to the memory in association with observation data;
the capturing circuitry inputs the observation data and the label data associated with the observation data that have been recorded to the memory, to the learning circuitry; and
the learning circuitry is characterized by using an activation function that outputs a predetermined ambiguous value in a process for determining a classification prediction result, and the learning circuitry performs learning using the inputted observation data and label data.

2. A learning device estimating method for use with a learning device, for classification task that outputs a type of inputted observation data as label data, as an attack target, the learning device estimating method using a learning device estimating apparatus comprising a memory, inquiring circuitry, capturing circuitry, and learning circuitry, the learning device estimating method comprising:
an inquiring step;
a capturing step; and
a learning step,
wherein:
the memory stores a predetermined plurality of pieces of observation data,
at the inquiring step, the inquiring circuitry inquires of the attack target learning device for each of the pieces of observation data recorded in the memory to acquire label data and records the acquired label data to the memory in association with observation data,
at the capturing step, the capturing circuitry inputs the observation data and the label data associated with the observation data that have been recorded to the memory, to the learning circuitry, and
at the learning step, the learning circuitry uses an activation function that outputs a predetermined ambiguous value in a process for determining a classification prediction result, and the learning circuitry performs learning using the inputted observation data and label data.

3. The learning device estimating method according to claim 2 , wherein:
the activation function that outputs the ambiguous value reduces a generalization error.

4. The learning device estimating method according to claim 2 , wherein:
a number of classified types is indicated by D (here, D is an integer equal to or larger than 2), T indicates a predetermined value equal to or larger than 1, c indicates an integer between 1 and D, including 1 and D, u c indicates the c-th element of a vector inputted to the activation function, and ˜ y c indicates the c-th element of a vector outputted as a classification result; and
the activation function is:

y
~

c

=

exp
⁡
(

u
c

/
T

)

∑

d
=
1

D

exp
⁡
(

u
c

/
T

)

.

5. A non-transitory computer readable medium storing a program for causing a computer to execute the learning device estimating method according to claim 2 .

6. A non-transitory computer readable medium storing a program for causing a computer to execute the learning device estimating method according to claim 3 .

7. A non-transitory computer readable medium storing a program for causing a computer to execute the learning device estimating method according to claim 4 .

8. A risk evaluation apparatus for evaluating a risk of an attack to a learning device for classification task that outputs a type of inputted observation data as label data, the risk evaluation apparatus comprising:
the learning device estimating apparatus according to claim 1 ;
correct answer rate acquiring circuitry determining a target correct answer rate, which is a correct answer rate of the learning device that has finished learning, and an estimated correct answer rate, which is a correct answer rate of the learning circuitry that has finished learning, using a predetermined plurality of pairs of observation data and label data for test; and
risk judging circuitry judging that a risk is higher as a difference between the target correct answer rate and the estimated correct answer rate is smaller when the target correct answer rate is larger than the estimated correct answer rate, and as the estimated correct answer rate exceeds the target correct answer rate more when the target correct answer rate is smaller than the estimated correct answer rate.

9. A risk evaluation method for evaluating a risk of an attack to a learning device for classification task that outputs a type of inputted observation data as label data, using a learning device estimating apparatus comprising a learning circuitry, the risk evaluation method comprising:
an attack target classification predicting step of inputting a plurality of pieces of observation data to the learning device that has finished learning, acquiring pieces of predicted label data that are classification predictions at the time of inputting the plurality of observation data, and obtaining a data set for estimation that is a set of pairs of observation data and predicted label data;
an estimation learning step of learning the learning circuitry using the data set for estimation to obtain the learning circuitry that has finished learning;
a correct answer rate acquiring step of determining a target correct answer rate, which is a correct answer rate of the learning device that has finished learning, and an estimated correct answer rate, which is a correct answer rate of the learning circuitry that has finished learning, using a predetermined plurality of pairs of observation data and label data for test; and
a risk judging step of judging that a risk is higher as a difference between the target correct answer rate and the estimated correct answer rate is smaller when the target correct answer rate is larger than the estimated correct answer rate, and as the estimated correct answer rate exceeds the target correct answer rate more when the target correct answer rate is smaller than the estimated correct answer rate,
wherein the learning circuitry uses an activation function that outputs a predetermined ambiguous value in a process for determining a classification prediction result.

10. The risk evalua

Assignees

• Nippon Telegraph & Telephone

Inventors

• Okada Rina
• Hasegawa Satoshi

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F21/577 Primary

  Assessing vulnerabilities and evaluating computer system security · CPC title

• G06F18/2155

  characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling · CPC title

• G06F18/24

  Classification techniques · CPC title