Rapid competitive learning techniques for neural networks

Patent metadata

Field	Value
Publication number	US-10846595-B2
Application number	US-201615385378-A
Country	US
Kind code	B2
Filing date	Dec 20, 2016
Priority date	Dec 20, 2016
Publication date	Nov 24, 2020
Grant date	Nov 24, 2020

Abstract

Various systems and methods for implementing unsupervised or reinforcement learning operations for a neuron weight used in a neural network are described. In an example, the learning operations include processing a spike train input at a neuron of a spiking neural network, applying a synaptic weight, and observing spike events occurring before and after the neuron processing based on respective spike traces. A synaptic weight update process operates to generate a new value of the synaptic weight based upon the spike traces, configuration values, and a reference weight value. A reference weight update process also operates to generate a new value of the reference value for significant changes to the synaptic weight. Reinforcement may be provided in some examples to implement changes to the reference weight in reduced time. In some examples, the techniques may be implemented in a neuromorphic hardware implementation of the spiking neural network.

First claim

What is claimed is:

1. At least one non-transitory machine readable medium including instructions for learning in a spiking neural network, the instructions, when executed by a machine, cause the machine to perform operations comprising:
processing an input at a neuron of the spiking neural network, wherein the processing includes the neuron applying a first value of a synaptic weight in response to at least one spike of a received spike train, and wherein the neuron causes propagation of at least one spike in a generated spike train from a synaptic connection in response to use of the synaptic weight on the received spike train;
generating a second value of the synaptic weight with a synaptic weight update process, wherein the synaptic weight update process evaluates values received from: the first value of the synaptic weight, a first value of a reference weight, at least one synaptic weight configuration parameter, at least one trace of the spikes in the received spike train, and at least one trace of the spikes in the generated spike train;
generating a second value of the reference weight with a reference weight update process, wherein the reference weight update process evaluates inputs received from: the first value of the synaptic weight, the first value of the reference weight, a time constant, and at least one reference weight configuration parameter; and
evaluating a difference between the first value of the synaptic weight and the first value of the reference weight;
wherein, in response to the difference exceeding a significance threshold, the second value of the reference weight is generated to change in a direction towards the first value of the synaptic weight; and
wherein, in response to the difference not exceeding the significance threshold, the second value of the synaptic weight is generated to change in a direction towards the first value of the reference weight.

2. The at least one non-transitory machine readable medium of claim 1 , the operations further comprising:
in response to a reinforcement signal, performing reinforcement learning in the reference weight update process, wherein the reinforcement learning in the reference weight update process includes reducing a value of the time constant and establishing the second value of the reference weight with use of the reduced value of the time constant, wherein the second value of the reference weight is updated to converge to the second value of the synaptic weight.

3. The at least one non-transitory machine readable medium of claim 2 , the operations further comprising:
in response to absence of the reinforcement signal, performing unsupervised learning in the synaptic weight update process, wherein the unsupervised learning in the synaptic weight update process includes utilizing the value of the time constant and establishing the second value of the synaptic weight by competitive unsupervised learning, wherein the second value of the synaptic weight converges to the first value of the reference weight.

4. The at least one non-transitory machine readable medium of claim 1 , the operations further comprising:
obtaining respective values of: the at least one synaptic weight configuration parameter, the at least one reference weight configuration parameter, and the time constant; and
wherein the time constant is initialized to an intermediate value.

5. The at least one non-transitory machine readable medium of claim 1 , wherein the reference weight update process operates to determine the second value of the reference weight based on changes to the reference weight over time, based on the time constant and the at least one reference weight configuration parameter.

6. The at least one non-transitory machine readable medium of claim 1 , wherein the synaptic weight update process operates to determine the second value of the synaptic weight based on weight normalization and a movement in a direction of first value of the reference weight, based on a drift force and a relaxation force.

7. The at least one non-transitory machine readable medium of claim 1 , wherein the spikes in the received spike train are provided from a first plurality of neurons in the spiking neural network, and wherein the spikes in the generated spike train are provided to a second plurality of neurons in the spiking neural network.

8. The at least one non-transitory machine readable medium of claim 1 , wherein the spiking neural network is provided by neuromorphic computing hardware having a plurality of cores, wherein respective cores of the plurality of cores are configurable to implement respective neurons used in the spiking neural network, and wherein spikes are used among the respective cores to communicate information on processing actions of the spiking neural network.

9. A computing device configured for implementing learning in a neuron of a spiking neural network, the computing device comprising circuitry to:
process an input at a neuron of the spiking neural network, wherein the processing includes the neuron applying a first value of a synaptic weight in response to at least one spike of a received spike train, and wherein the neuron causes propagation of at least one spike in a generated spike train from a synaptic connection in response to use of the synaptic weight on the received spike train;
generate a second value of the synaptic weight with a synaptic weight update process, wherein the synaptic weight update process evaluates values received from: the first value of the synaptic weight, a first value of a reference weight, at least one synaptic weight configuration parameter, at least one trace of the spikes in the received spike train, and at least one trace of the spikes in the generated spike train;
generate a second value of the reference weight with a reference weight update process, wherein the reference weight update process evaluates inputs received from: the first value of the synaptic weight, the first value of the reference weight, a time constant, and at least one reference weight configuration parameter; and
evaluate a difference between the first value of the synaptic weight and the first value of the reference weight;
wherein, in response to the difference exceeding a significance threshold, the second value of the reference weight is generated to change in a direction towards the first value of the synaptic weight; and
wherein, in response to the difference not exceeding the significance threshold, the second value of the synaptic weight is generated to change in a direction towards the first value of the reference weight.

10. The computing device of claim 9 , the circuitry further to:
perform reinforcement learning in the reference weight update process, in response to a reinforcement signal, wherein the reinforcement learning in the reference weight update process includes reduction of a value of the time constant and establishment of the second value of the reference weight with use of the reduced value of the time constant, wherein the second value of the reference weight is updated to converge to the second value of the synaptic weight.

11. The computing device of claim 10 , the circuitry further to:
perform unsupervised learning in the synaptic weight update process, in response to absence of the reinforcement signal, wherein the unsupervised learning in the synaptic weight update process includes use of the value of the time constant and establishment of the second value of the synaptic weight by competitive unsupervised learning, wherein the second value of the synaptic weight converges to the first value of the reference weight.

12. The computing device of claim 9 , the circuitry

Assignees

• Intel Corp

Inventors

• Wild Andreas
• Srinivasa Narayan

Classifications

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06N3/088 Primary

  Non-supervised learning, e.g. competitive learning · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title