Device, system and method for varying a synaptic weight with a phase differential of a spiking neural network

Patent metadata

Field	Value
Publication number	US-11568241-B2
Application number	US-201716648437-A
Country	US
Kind code	B2
Filing date	Dec 19, 2017
Priority date	Dec 19, 2017
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

Techniques and mechanisms for determining the value of a weight associated with a synapse of a spiking neural network. In an embodiment, a first spike train and a second spike train are output, respectively, by a first node and a second node of the spiking neural network, wherein the synapse is coupled between said nodes. The weight is applied to signaling communicated via the synapse. A value of the weight is updated based on a product of a first value and a second value, wherein the first value is based on a first rate of spiking by the first spike train, and the second value is based on a second rate of spiking by the second spike train. In another embodiment, the weight is updated based on a product of a derivative of the first rate of spiking and a derivative of the second rate of spiking.

First claim

What is claimed is:

1. A computer device for training a spiking neural network to recognize a data type, the computer device comprising circuitry to:
communicate a first generated spike train from a first node of a spiking neural network, the first generated spike train based on one or more signal spikes of a first received spike train provided to the first node, wherein the first generated spike train exhibits a first rate of spiking;
communicate a second generated spike train from a second node of the spiking neural network, the second generated spike train based on one or more signal spikes of a second received spike train provided to the second node, wherein the second generated spike train exhibits a second rate of spiking, and wherein a synapse is directly coupled to each of the first node and the second node;
apply a first value of a synaptic weight to at least one signal spike communicated via the synapse; and
determine a second value of the synaptic weight, including circuitry to signal a change to apply to the first value of the synaptic weight, the change based on a product of a first value based on the first rate of spiking and a second value based on the second rate of spiking, wherein a training of the spiking neural network is based on the change being applied to the first value.

2. The computer device of claim 1 , wherein the change is based on a product of a first derivative of the first rate of spiking and a second derivative of the second rate of spiking.

3. The computer device of claim 1 , wherein the first derivative and the second derivative each include a respective one of a first order derivative and a second order derivative.

4. The computer device of claim 1 , wherein the first node is to send the first generated spike train to the second node via the synapse.

5. The computer device of claim 1 , wherein the first generated spike train and the second generated spike train are each to be communicated via a respective path which is independent of the synapse.

6. The computer device of claim 1 , wherein the second generated spike train is to be determined based on the first generated spike train.

7. The computer device of claim 1 , wherein the first generated spike train and the second generated spike train are each to be determined based on a different respective spike train.

8. The computer device of claim 1 , further comprising circuitry to select a first subset of nodes of the spiking neural network to train the spike neural network, during a first time period, to determine updates to weights of respective synapses.

9. The computer device of claim 8 , further comprising circuitry to select a second subset of nodes of the spiking neural network to train the spike neural network, during a second time period, to determine updates to weights of respective synapses.

10. At least one non-transitory machine readable medium including instructions that, when executed by a machine, cause the machine to perform operations for training a spiking neural network to recognize a data type, the operations comprising:
communicating a first generated spike train from a first node of a spiking neural network, the first generated spike train based on one or more signal spikes of a first received spike train provided to the first node, wherein the first generated spike train exhibits a first rate of spiking;
communicating a second generated spike train from a second node of the spiking neural network, the second generated spike train based on one or more signal spikes of a second received spike train provided to the second node, wherein the second generated spike train exhibits a second rate of spiking, and wherein a synapse is directly coupled to each of the first node and the second node;
applying a first value of a synaptic weight to at least one signal spike communicated via the synapse; and
determining a second value of the synaptic weight, including signaling a change to apply to the first value of the synaptic weight, the change based on a product of a first value based on the first rate of spiking and a second value based on the second rate of spiking, wherein a training of the spiking neural network is based on the change being applied to the first value.

11. The at least one non-transitory machine readable medium of claim 10 , wherein the change is based on a product of a first derivative of the first rate of spiking and a second derivative of the second rate of spiking.

12. The at least one non-transitory machine readable medium of claim 10 , wherein the first derivative and the second derivative each include a respective one of a first order derivative and a second order derivative.

13. The at least one non-transitory machine readable medium of claim 10 , wherein the first node sends the first generated spike train to the second node via the synapse.

14. The at least one non-transitory machine readable medium of claim 10 , wherein the first generated spike train and the second generated spike train are each communicated via a respective path which is independent of the synapse.

15. The at least one non-transitory machine readable medium of claim 10 , wherein the second generated spike train is determined based on the first generated spike train.

16. The at least one non-transitory machine readable medium of claim 10 , wherein the first generated spike train and the second generated spike train are each determined based on a different respective spike train.

17. The at least one non-transitory machine readable medium of claim 10 , the operations further comprising selecting a first subset of nodes of the spiking neural network to train the spike neural network, during a first time period, to determine updates to weights of respective synapses.

18. The at least one non-transitory machine readable medium of claim 17 , the operations further comprising selecting a second subset of nodes of the spiking neural network to train the spike neural network, during a second time period, to determine updates to weights of respective synapses.

19. A method for training a spiking neural network to recognize a data type, the method comprising:
communicating a first generated spike train from a first node of a spiking neural network, the first generated spike train based on one or more signal spikes of a first received spike train provided to the first node, wherein the first generated spike train exhibits a first rate of spiking;
communicating a second generated spike train from a second node of the spiking neural network, the second generated spike train based on one or more signal spikes of a second received spike train provided to the second node, wherein the second generated spike train exhibits a second rate of spiking, and wherein a synapse is directly coupled to each of the first node and the second node;
applying a first value of a synaptic weight to at least one signal spike communicated via the synapse; and
determining a second value of the synaptic weight, including signaling a change to apply to the first value of the synaptic weight, the change based on a product of a first value based on the first rate of spiking and a second value based on the second rate of spiking, wherein a training of the spiking neural network is based on the change being applied to the first value.

20. The method of claim 19 , wherein the change is based on a product of a first derivative of the first rate of spiking and a second derivative of the second rate of spiking.

21. The method of claim 19 , wherein the

Assignees

• Intel Corp

Inventors

• Paul Arnab
• Srinivasa Narayan

Classifications

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title