Information processing apparatus, information processing method, and computer-readable storage medium

Patent metadata

Field	Value
Publication number	US-2019362232-A1
Application number	US-201916532812-A
Country	US
Kind code	A1
Filing date	Aug 6, 2019
Priority date	Mar 31, 2017
Publication date	Nov 28, 2019
Grant date	—

Abstract

An information processing apparatus includes a weight setting unit configured to set a plurality of weights of a selection layer selected from a plurality of layers of a first neural network as a plurality of weights of a second neural network; a classification unit configured to classify each of the weights of the selection layer into a first group or a second group; a first determination unit configured to determine a first gradient for each weight of the first neural network, based on first training data; a second determination unit configured to determine a second gradient for weights belonging to the first group based on second training data; and an updating unit configured to update the weights belonging to the first group based on the first gradient and the second gradient, and updating the other weights based on the first gradient.

First claim

1 . An information processing apparatus comprising:
a weight setting unit configured to set a plurality of weights of a selection layer selected from a plurality of layers of a first neural network as a plurality of weights of a second neural network; a classification unit configured to classify each of the plurality of weights of the selection layer into a first group or a second group; a first determination unit configured to determine a first gradient for each weight of the plurality of layers of the first neural network, based on first training data; a second determination unit configured to determine a second gradient for weights belonging to the first group among the plurality of weights of the second neural network, based on second training data; and an updating unit configured to update the weights belonging to the first group, among the plurality of weights of the selection layer, based on the first gradient determined by the first determination unit and the second gradient determined by the second determination unit, and updating the weights belonging to the second group, among the plurality of weights of the selection layer, and weights of the layers other than the selection layer among the plurality of layers of the first neural network, based on the first gradient determined by the first determination unit.

2 . The information processing apparatus according to claim 1 ,
wherein the first determination unit sets the first gradient to 0 for the weights belonging to the first group, among the plurality of weights of the selection layer of the first neural network.

3 . The information processing apparatus according to claim 1 ,
wherein the classification unit classifies each of the plurality of weights of the selection layer into the first group or the second group according to the second training data, and the weights of the selection layer that are shown by the first group do not overlap with respect to different pieces of the second training data.

4 . The information processing apparatus according to claim 1 ,
wherein the weight setting unit preferentially selects a layer close to an input layer of the first neural network, among the plurality of layers of the first neural network, as the selection layer.

5 . The information processing apparatus according to claim 1 ,
wherein the second determination unit determines the second gradient for the weights belonging to the second group, among the plurality of weights of the second neural network, with a value of said weights set to 0.

6 . An information processing method according to which a processor executes:
selecting a selection layer from a plurality of layers of a first neural network; setting the selection layer as a layer constituting a second neural network; classifying each of a plurality of weights of the selection layer into a first group or a second group; determining a first gradient for each weight of the plurality of layers of the first neural network, based on first training data; determining a second gradient for weights belonging to the first group, among the plurality of weights of the selection layer constituting the second neural network, based on second training data; and updating the weights belonging to the first group, among the plurality of weights of the selection layer, based on the first gradient and the second gradient, and updating the weights belonging to the second group, among the plurality of weights of the selection layer, and weights of the layers other than the selection layer among the plurality of layers of the first neural network, based on the first gradient.

7 . A computer-readable storage medium storing a program, the program, when executed by one or more processors, causing the one or more processors to execute:
selecting a selection layer from a plurality of layers of a first neural network; setting the selection layer as a layer constituting a second neural network; classifying each of a plurality of weights of the selection layer into a first group or a second group; determining a first gradient for each weight of the plurality of layers of the first neural network, based on first training data; determining a second gradient for weights belonging to the first group, among the plurality of weights of the selection layer constituting the second neural network, based on second training data; and updating the weights belonging to the first group, among the plurality of weights of the selection layer, based on the first gradient and the second gradient, and updating the weights belonging to the second group, among the plurality of weights of the selection layer, and weights of the layers other than the selection layer among the plurality of layers of the first neural network, based on the first gradient.

Assignees

• Kddi Corp

Inventors

• Nagai Yuki
• Kobayashi Tatsuya

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F21/16

  Program or content traceability, e.g. by watermarking · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title