Computer system and model training method

Patent metadata

Field	Value
Publication number	US-12314848-B2
Application number	US-202016939464-A
Country	US
Kind code	B2
Filing date	Jul 27, 2020
Priority date	Aug 13, 2019
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

Provided is a computer system for generating a neural network (NN) used for a task including a training unit that calculates a weighting factor between a reservoir and an output layer using training data, a first storage unit that stores, as node activity information, information on a node activity level of a node included in the reservoir, and a second storage unit that stores model information. When receiving a training execution request for a second task after a training process of a first task is completed, the training unit calculates a weighting factor of the NN used for a third task obtained by combining the first task and the second task, updates the model information based on the calculated weighting factor, and updates the node activity information based on the node activity measured during execution of the training process of the third task.

First claim

What is claimed is:

1. A computer system that generates, as a model, a recurrent neural network used for a task of obtaining an output result for input data to be processed, the computer system comprising:
at least one computer having a processor and a storage device connected to the processor,
wherein the recurrent neural network includes an input layer, a reservoir, and an output layer,
wherein the input layer includes a first node that receives a plurality of time-series data,
wherein the reservoir receives an output from the input layer, and includes a plurality of second nodes that form a recurrent network,
wherein the output layer includes third nodes that receive an output from the reservoir,
wherein the processor is configured to:
execute a training process of calculating a weighting factor indicating a strength of a connection between the second nodes and the third nodes using input data including a value of at least one component and training data including a target output result including a value of at least one component,
store, as node activity information, information of node activity that is an output value of each of the plurality of second nodes measured during execution of the training process of a task,
store model information that defines a structure of the recurrent neural network,
in response to receiving a training execution request for a second task after completion of the training process of a first task, calculate the weighting factor of the recurrent neural network used for a third task, which is a combination of the first task and the second task, based on the training data of the second task, the node activity information, and the model information as a first process, that includes calculating a first value from a first equation

Task
y

=

1
N

⁢

∑
t

X
T

⁢
y

wherein N represents a number of the training data, X represents an n×n matrix of node activity, n being an integer greater than 1, T represents a transposed matrix X, and Σ represents a summation of time span of the training data of a corresponding task, based on the target output result, time-series data of node activity of each of the plurality of second nodes measured during the execution of the training process of the third task, and the node activity information, and calculating a second value from a second equation

Task
X

=

1
N

⁢

(

∑
t

X
T

⁢
X

)

-
1

based on the time-series data of the node activity of each of the plurality of second nodes measured during the execution of the training process of the third task, and the node activity information,
update the stored model information based on the calculated weighting factor as a second process, and
update the stored node activity information based on the first value from the first equation, the second value from the second equation, and the node activity of each of the second nodes measured during execution of the training process of the third task as a third process.

2. The computer system according to claim 1 ,
wherein, when executing the training process of the third task, the processor is configured to set a number of third nodes in the output layer as a same number as a sum of components included in the target output result of the first task and components included in the target output result of the second task.

3. The computer system according to claim 2 ,
calculate the weighting factor based on the first value and the second value, and
update the node activity information based on the number of the input data input in the training process of the third task.

4. The computer system according to claim 2 ,
wherein the processor is configured to:
update the target output result by adding a component to the input data to the target output result, and
execute the training process of the third task using the training data including the input data and the updated target output result.

5. A model training method, executed by a computer system, for generating, as a model, a recurrent neural network used for a task of obtaining an output result for input data to be processed,
wherein the computer system comprises at least one computer having a processor and a storage device connected to the processor,
wherein the recurrent neural network includes an input layer, a reservoir, and an output layer,
wherein the input layer includes a first node that receives a plurality of time-series data,
wherein the reservoir receives an output from the input layer, and includes a plurality of second nodes that form a recurrent network,
wherein the output layer includes third nodes that receive an output from the reservoir, the method comprising:
executing a training process of calculating a weighting factor indicating a strength of a connection between the second nodes and the third nodes using input data including a value of at least one component and training data including a target output result including a value of at least one component;
storing, as node activity information, information of node activity that is an output value of each of the plurality of second nodes measured during execution of the training process of a task;
storing model information that defines a structure of the recurrent neural network,
in response to receiving a training execution request for a second task after completion of the training process of a first task, calculating the weighting factor of the recurrent neural network used for a third task, which is a combination of the first task and the second task, with the recurrent neural network based on the training data of the second task, the node activity information, and the model information as a first step, that includes calculating a first value from a first equation

Task
y

=

1
N

⁢

∑
t

X
T

⁢
y

wherein N represents a numbe

Assignees

• Hitachi Ltd

Inventors

• Okumura Tadashi
• NAGATA Sanato
• Ando Masahiko

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title