Computer-readable recording medium storing speed-up program, speed-up method, and information processing device

Patent metadata

Field	Value
Publication number	US-2023083790-A1
Application number	US-202217844765-A
Country	US
Kind code	A1
Filing date	Jun 21, 2022
Priority date	Sep 10, 2021
Publication date	Mar 16, 2023
Grant date	—

Abstract

A computer-implemented method of a speed-up processing, the method including: calculating variance of weight information regarding a weight updated by machine learning, for each layer included in a machine learning model at a predetermined interval at time of the machine learning of the machine learning model; and determining a suppression target layer that suppresses the machine learning on the basis of a peak value of the variance calculated at the predetermined interval and the variance of the weight information calculated at the predetermined interval.

First claim

What is claimed is:

1 . A non-transitory computer-readable recording medium storing a speed-up program for causing a computer to execute processing comprising:
calculating variance of weight information regarding a weight updated by machine learning, for each layer included in a machine learning model at a predetermined interval at time of the machine learning of the machine learning model; and determining a suppression target layer that suppresses the machine learning on the basis of a peak value of the variance calculated at the predetermined interval and the variance of the weight information calculated at the predetermined interval.

2 . The non-transitory computer-readable recording medium according to claim 1 , wherein
the processing of calculating includes calculating the variance of the weight information for the each layer, for each iteration as the predetermined interval, and the processing of determining includes specifying, for the each layer, a waveform pattern of the variance of the weight information on the basis of a tendency of the variance of the weight information calculated for the each iteration, setting a threshold according to the waveform pattern for the each layer, and determining a layer in which a difference between the peak value of the variance and the variance of the weight information calculated at the predetermined interval is equal to or larger than the threshold, as the suppression target layer.

3 . The non-transitory computer-readable recording medium according to claim 1 , wherein
the processing of determining includes selecting a target layer that determines the suppression target layer in order from an input layer of the machine learning model, and determining, for the target layer, whether to suppress the machine learning for the each predetermined interval on the basis of the peak value of the variance and the variance of the weight information.

4 . The non-transitory computer-readable recording medium according to claim 3 , wherein
the processing of calculating starts the calculation of the variance of the weight information for a second layer located next to a first layer determined as the suppression target layer after the machine learning of the first layer is stopped, of a plurality of layers included in the machine learning model.

5 . The non-transitory computer-readable recording medium according to claim 1 , wherein
the processing of calculating includes dividing a plurality of layers included in the machine learning model into a plurality of blocks in order from an input layer, and calculating the variance of the weight information for each layer that belongs to each of the plurality of blocks, and the processing of determining includes selecting a target block that determines the suppression target layer in order close to an input of the machine learning model, and suppressing, for the target block, the machine learning of each layer that belongs to the target block in a case where a difference between the peak value of the variance of the each layer that belongs to the target block and the variance of the weight information becomes equal to or larger than a threshold.

6 . The non-transitory computer-readable recording medium according to claim 5 , wherein
the processing of calculating starts the calculation of the variance of the weight information for a second block located next to a first block determined as the suppression target layer after the machine learning of the first block is stopped, of the plurality of block layers.

7 . The non-transitory computer-readable recording medium according to claim 1 , wherein
the processing of determining decreases a learning rate for each iteration and executes the machine learning during a fixed period until the machine learning is suppressed, for the suppression target layer that suppresses the machine learning.

8 . A computer-implemented method of a speed-up processing, the method comprising:
calculating variance of weight information regarding a weight updated by machine learning, for each layer included in a machine learning model at a predetermined interval at time of the machine learning of the machine learning model; and determining a suppression target layer that suppresses the machine learning on the basis of a peak value of the variance calculated at the predetermined interval and the variance of the weight information calculated at the predetermined interval.

9 . An information processing apparatus comprising:
a memory; and a processor coupled to the memory, the processor being configured to perform processing, the processing including: calculating variance of weight information regarding a weight updated by machine learning, for each layer included in a machine learning model at a predetermined interval at time of the machine learning of the machine learning model; and determining a suppression target layer that suppresses the machine learning on the basis of a peak value of the variance calculated at the predetermined interval and the variance of the weight information calculated at the predetermined interval.

Assignees

• Fujitsu Ltd

Inventors

• Hara Yasushi
• Kasagi Akihiko
• Kai Yutaka
• Danjo Takumi

Classifications

• G06N3/063

  using electronic means · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title