Adaptively training of neural networks via an intelligent learning management system

Patent metadata

Field	Value
Publication number	US-12579408-B2
Application number	US-202519236733-A
Country	US
Kind code	B2
Filing date	Jun 12, 2025
Priority date	Aug 20, 2020
Publication date	Mar 17, 2026
Grant date	Mar 17, 2026

Abstract

Computer systems and computer-implemented methods train a neural network, by:
(a) computing for each datum in a set of training data, activation values for nodes in the neural network and estimates of partial derivatives of an objective function for the neural network for the nodes in the neural network; (b) selecting a target node of the neural network and/or a target datum in the set of training data; (c) selecting a target-specific improvement model for the neural network, wherein the target-specific improvement model, when added to the neural network, improves performance of the neural network for the target node and/or the target datum, as the case may be; (d) training the target-specific improvement model; (e) merging the target-specific improvement model with the neural network to form an expanded neural network; and (f) training the expanded neural network.

First claim

What is claimed is:

1 . A computer system for adaptively training a neural network, wherein:
the neural network is for performing a machine-learning task; the neural network comprises an input layer, an output layer, and one or more inner layers; and each layer of the neural network comprises at least one node, the computer system comprising one or more processor units; and a memory in communication with the one or more processor units, the memory storing computer instructions that, when executed by the one or more processor units, cause the computer system to:
train, at least partially, via machine learning, the neural network on a set of training data;
identify, based on performance of the machine-learning task by the neural network, a target node of the neural network and a target datum for improving performance of the neural network on the machine-learning task;
determine, via an intelligent learning management system (ILMS) configured to receive and process human feedback, whether a provider node exhibits a human-interpretable activation pattern for the target datum that the ILMS identifies as human-interpretable based on the human feedback;
in response to determining that the provider node exhibits a human-interpretable activation pattern for the target datum, selectively apply a relationship link between the target node and the provider node based on interpretability and performance criteria; and
retrain the neural network with the relationship link applied, thereby modifying a behavior of the target node for the target datum,
wherein the ILMS enables interpretable modification of internal representations of the neural network while preserving the neural network's task performance.

2 . The computer system of claim 1 , wherein the relationship link comprises a node-to-node regularization link that imposes a regularization cost during training based on a difference between activation values of the target node and the provider node for the target datum.

3 . The computer system of claim 2 , wherein the regularization cost comprises a squared error loss computed as the square of a difference between the activation value of the target node and the activation value of the provider node.

4 . The computer system of claim 1 , wherein selectively applying the relationship link comprises storing an indication of the relationship link in the memory, associating the target node with the provider node.

5 . The computer system of claim 1 , wherein the ILMS comprises a cooperative human-AI learning supervisor system that integrates human interpretability feedback with automated performance evaluation for link selection.

6 . The computer system of claim 5 , wherein the human interpretability feedback comprises a human-labeled judgment that the activation pattern of the provider node corresponds to a known interpretable concept.

7 . The computer system of claim 1 , wherein the relationship link is applied only upon a determination that the provider node has previously been identified as interpretable across a plurality of data examples.

8 . The computer system of claim 2 , wherein retraining the neural network with the relationship link includes modifying a weight associated with the target node based on the regularization cost imposed by the relationship link.

9 . The computer system of claim 1 , wherein the ILMS further prevents application of a relationship link in response to determining that the activation pattern of the target node is non-interpretable.

10 . The computer system of claim 1 , wherein the performance criteria comprise a threshold accuracy or loss metric associated with the machine-learning task.

11 . The computer system of claim 1 , wherein the ILMS further enables a human to override a selected relationship link and define a substitute provider node for the target node based on interpretability feedback.

12 . The computer system of claim 2 , wherein the node-to-node regularization link is applied only between nodes located in a selected inner layer of the neural network.

13 . A computer-implemented method for adaptively training a neural network, the method comprising:
training, by a programmed computer system, at least partially, a neural network on a set of training data, wherein the neural network is configured to perform a machine-learning task and comprises an input layer, an output layer, and one or more inner layers, each layer comprising at least one node; identifying, by the programmed computer system and based on performance of the machine-learning task by the neural network, a target node of the neural network and a target datum for improving performance of the neural network on the machine-learning task; determining, by the programmed computer system via an intelligent learning management system (ILMS) configured to receive and process human feedback, whether a provider node exhibits a human-interpretable activation pattern for the target datum that the ILMS identifies as human-interpretable based on the human feedback; in response to determining that the provider node exhibits a human-interpretable activation pattern for the target datum, selectively applying, by the programmed computer system, a relationship link between the target node and the provider node based on interpretability and performance criteria; and retraining, by the programmed computer system, the neural network with the relationship link applied, thereby modifying a behavior of the target node for the target datum, wherein the ILMS enables interpretable modification of internal representations of the neural network while preserving the neural network's task performance.

14 . The method of claim 13 , wherein the relationship link comprises a node-to-node regularization link that imposes a regularization cost during training based on a difference between activation values of the target node and the provider node for the target datum.

15 . The method of claim 14 , wherein the regularization cost comprises a squared error loss computed as the square of a difference between the activation value of the target node and the activation value of the provider node.

16 . The method of claim 13 , wherein selectively applying the relationship link comprises storing, by the programmed computer system in memory, an indication of the relationship link that associates the target node with the provider node.

17 . The method of claim 13 , wherein the ILMS comprises a cooperative human-AI learning supervisor system that integrates human interpretability feedback with automated performance evaluation for link selection.

18 . The method of claim 17 , wherein the human interpretability feedback comprises a human-labeled judgment that the activation pattern of the provider node corresponds to a known interpretable concept.

19 . The method of claim 13 , wherein the relationship link is applied only upon a determination, by the programmed computer system, that the provider node has previously been identified as interpretable across a plurality of data examples.

20 . The method of claim 14 , wherein retraining the neural network with the relationship link includes modifying, by the programmed computer system, a weight associated with the target node based on the regularization cost imposed by the relationship link.

21 . The method of claim 13 , wherein the ILMS further prevents application of a relationship link in response to determining that the activation pattern of the target node i

Assignees

• D5Ai Llc

Inventors

• Baker James K
• Baker Bradley J

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title