Information processing apparatus, information processing method, and computer-readable storage medium

Patent metadata

Field	Value
Publication number	US-11461641-B2
Application number	US-201916532812-A
Country	US
Kind code	B2
Filing date	Aug 6, 2019
Priority date	Mar 31, 2017
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

An information processing apparatus includes a weight setting unit configured to set a plurality of weights of a selection layer selected from a plurality of layers of a first neural network as a plurality of weights of a second neural network; a classification unit configured to classify each of the weights of the selection layer into a first group or a second group; a first determination unit configured to determine a first gradient for each weight of the first neural network, based on first training data; a second determination unit configured to determine a second gradient for weights belonging to the first group based on second training data; and an updating unit configured to update the weights belonging to the first group based on the first gradient and the second gradient, and updating the other weights based on the first gradient.

First claim

The invention claimed is:

1. An information processing apparatus comprising:
at least one processor circuit with a memory comprising instructions, that when executed by the processor circuit, causes the at least one processor circuit to at least:
set a plurality of weights of a selection layer selected from a plurality of layers of a first neural network as a plurality of weights of a second neural network;
classify each of the plurality of weights of the selection layer into a first group or a second group;
determine a first gradient for each weight of the plurality of layers of the first neural network, based on first training data;
determine a second gradient for weights belonging to the first group among the plurality of weights of the second neural network, based on second training data; and
update the weights belonging to the first group, among the plurality of weights of the selection layer, based on the first gradient and the second gradient and updating the weights belonging to the second group, among the plurality of weights of the selection layer, and weights of the layers other than the selection layer among the plurality of layers of the first neural network, based on the first gradient;

wherein each of the plurality of weights of the selection layer into the first group or the second group is classified according to the second training data; and
the weights of the selection layer that are shown by the first group do not overlap with respect to different pieces of the second training data.

2. The information processing apparatus according to claim 1 ,
wherein the first gradient is set to 0 for the weights belonging to the first group, among the plurality of weights of the selection layer of the first neural network.

3. The information processing apparatus according to claim 1 ,
wherein a layer close to an input layer of the first neural network is preferentially selected, among the plurality of layers of the first neural network, as the selection layer.

4. The information processing apparatus according to claim 1 ,
wherein the second gradient is determined for the weights belonging to the second group, among the plurality of weights of the second neural network, with a value of said weights set to 0.

5. An information processing method according to which a processor having a memory executes:
selecting a selection layer from a plurality of layers of a first neural network;
setting the selection layer as a layer constituting a second neural network;
classifying each of a plurality of weights of the selection layer into a first group or a second group;
determining a first gradient for each weight of the plurality of layers of the first neural network, based on first training data;
determining a second gradient for weights belonging to the first group, among the plurality of weights of the selection layer constituting the second neural network, based on second training data; and
updating the weights belonging to the first group, among the plurality of weights of the selection layer, based on the first gradient and the second gradient, and updating the weights belonging to the second group, among the plurality of weights of the selection layer, and weights of the layers other than the selection layer among the plurality of layers of the first neural network, based on the first gradient
wherein each of the plurality of weights of the selection layer into the first group or the second group is classified according to the second training data; and
the weights of the selection layer that are shown by the first group do not overlap with respect to different pieces of the second training data.

6. A non-transitory computer-readable storage medium storing a program, the program, when executed by one or more processors, causing the one or more processors to execute:
selecting a selection layer from a plurality of layers of a first neural network;
setting the selection layer as a layer constituting a second neural network;
classifying each of a plurality of weights of the selection layer into a first group or a second group;
determining a first gradient for each weight of the plurality of layers of the first neural network, based on first training data;
determining a second gradient for weights belonging to the first group, among the plurality of weights of the selection layer constituting the second neural network, based on second training data; and
updating the weights belonging to the first group, among the plurality of weights of the selection layer, based on the first gradient and the second gradient, and updating the weights belonging to the second group, among the plurality of weights of the selection layer, and weights of the layers other than the selection layer among the plurality of layers of the first neural network, based on the first gradient;
wherein each of the plurality of weights of the selection layer into the first group or the second group is classified according to the second training data; and
the weights of the selection layer that are shown by the first group do not overlap with respect to different pieces of the second training data.

Assignees

• Kddi Corp

Inventors

• Nagai Yuki
• Kobayashi Tatsuya

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title