Time-division multiplexed neurosynaptic module with implicit memory addressing for implementing a neural network

Patent metadata

Field	Value
Publication number	US-10331998-B2
Application number	US-201514963133-A
Country	US
Kind code	B2
Filing date	Dec 8, 2015
Priority date	Dec 21, 2012
Publication date	Jun 25, 2019
Grant date	Jun 25, 2019

Abstract

Embodiments of the invention relate to a time-division multiplexed neurosynaptic module with implicit memory addressing for implementing a neural network. One embodiment comprises maintaining neuron attributes for multiple neurons and maintaining incoming firing events for different time steps. For each time step, incoming firing events for said time step are integrated in a time-division multiplexing manner. Incoming firing events are integrated based on the neuron attributes maintained. For each time step, the neuron attributes maintained are updated in parallel based on the integrated incoming firing events for said time step.

First claim

What is claimed is:

1. A method comprising:
multiplexing computation and control logic for multiple neurons utilizing a processor including a memory array and a computation logic circuit, wherein the multiplexing comprises:
maintaining, in the memory array, synaptic connectivity information for the multiple neurons, wherein the memory array comprises multiple entries, and each entry of the memory array maintains a synaptic weight for a corresponding neuron of the multiple neurons; and
integrating, via the computation logic circuit, one or more firing events targeting the multiple neurons, wherein the computation logic circuit is time-multiplexed based on a number of neurons included in the multiple neurons, and the integrating comprises:
at a first time step of multiple time steps:
accessing a collection of axon vectors, wherein each axon vector of the collection corresponds to a time step of the multiple time steps, the axon vector comprises multiple bits, each bit of the axon vector corresponds to an axon of multiple axons connected to the multiple neurons, and the bit indicates whether a firing event is scheduled for delivery to the corresponding axon in the corresponding time step;
reading a first axon vector from the collection that corresponds to the first time step; and
for a first bit of the first axon vector that indicates a first firing event is scheduled for delivery to a corresponding first axon of the multiple axons in the first time step:
 reading a first synaptic weight for a first neuron of the multiple neurons that is connected to the corresponding first axon from a first entry of the memory array that corresponds to the first neuron;
 generating first neuronal firing activity based on the first synaptic weight, wherein the first neuronal firing activity comprises a second firing event targeting a second axon of the multiple axons in a second time step of the multiple time steps, and the second time step occurs after the first time step;
 writing an updated synaptic weight for the first neuron to the first entry of the memory array, wherein the updated synaptic weight is based on at least one of a learning rule and the first neuronal firing activity; and
 updating a second axon vector from the collection that corresponds to the second time step to include information identifying the second firing event for delivery to the second axon in the second time step.

2. The method of claim 1 , wherein the multiplexing further comprises:
dividing each time step into multiple time slots, wherein a number of time slots included in the multiple time slots is equal to the number of neurons, each time slot corresponds to a neuron of the multiple neurons, and the number of neurons is based in part on performance of the memory array.

3. The method of claim 1 , wherein the multiplexing integrating further comprises:
at the first time step:
delivering, via the computation logic circuit, the first firing event to the first axon based on the first axon vector.

4. The method of claim 3 , wherein the integrating further comprises:
at the first time step:
integrating the first firing event into a membrane potential variable of the first neuron based on the first synaptic weight.

5. The method of claim 4 , wherein the integrating further comprises:
at the first time step:
the first neuronal firing activity in response to the membrane potential variable of the first neuron exceeding a threshold parameter of the first neuron;
generating an update vector comprising multiple bits, wherein each bit of the update vector corresponds to a neuron of the multiple neurons and indicates whether the computation logic circuit generated neuronal firing activity for the corresponding neuron during the time step, and one or more axon vectors from the collection are updated based on the update vector; and
controlling transposable access to the memory array for enhanced learning operation, wherein one or more synaptic weights are readable from and writable to one or more entries of the memory array in both horizontal and vertical directions.

6. A system comprising a computer processor, a computer-readable hardware storage medium, and program code embodied with the computer-readable hardware storage medium for execution by the computer processor to implement a method comprising:
multiplexing computation and control logic for multiple neurons utilizing a processor including a memory array and a computation logic circuit, wherein the multiplexing comprises:
maintaining, in the memory array, synaptic connectivity information for the multiple neurons, wherein the memory array comprises multiple entries, and each entry of the memory array maintains a synaptic weight for a corresponding neuron of the multiple neurons; and
integrating, via the computation logic circuit, one or more firing events targeting the multiple neurons, wherein the computation logic circuit is time-multiplexed based on a number of neurons included in the multiple neurons, and the integrating comprises:
at a first time step of multiple time steps:
accessing a collection of axon vectors, wherein each axon vector of the collection corresponds to a time step of the multiple time steps, the axon vector comprises multiple bits, each bit of the axon vector corresponds to an axon of multiple axons connected to the multiple neurons, and the bit indicates whether a firing event is scheduled for delivery to the corresponding axon in the corresponding time step;
reading a first axon vector from the collection that corresponds to the first time step; and
for a first bit of the first axon vector that indicates a first firing event is scheduled for delivery to a corresponding first axon of the multiple axons in the first time step:
 reading a first synaptic weight for a first neuron of the multiple neurons that is connected to the corresponding first axon from a first entry of the memory array that corresponds to the first neuron;
 generating first neuronal firing activity based on the first synaptic weight, wherein the first neuronal firing activity comprises a second firing event targeting a second axon of the multiple axons in a second time step of the multiple time steps, and the second time step occurs after the first time step;
 writing an updated synaptic weight for the first neuron to the first entry of the memory array, wherein the updated synaptic weight is based on at least one of a learning rule and the first neuronal firing activity; and
 updating a second axon vector from the collection that corresponds to the second time step to include information identifying the second firing event for delivery to the second axon in the second time step.

7. The system of claim 6 , wherein the multiplexing further comprises:
dividing each time step into multiple time slots, wherein a number of time slots included in the multiple time slots is equal to the number of neurons, each time slot corresponds to a neuron of the multiple neurons, and the number of neurons is based in part on performance of the memory array.

8. The system of claim 6 , wherein the integrating further comprises:
at the first time step:
delivering, via the computation logic circuit, the first firing event to the first axon based on the first axon vector.

9. The system of claim 8 , wherein the integrating further comprises:
at the first time step:
integrating the first firing event into a membrane potential variable of the first neuron based on the first synaptic weight.

10. The system of claim 9 , wherein the integrating further comprises:
at the first time step:
the first neuronal firing activity in response to the membrane potential variable of the first neuron e

Assignees

• IBM

Inventors

• Arthur John V
• Brezzo Bernard V
• Chang Leland
• Friedman Daniel J
• Merolla Paul A
• Modha Dharmendra S
• Montoye Robert K
• Seo Jae-Sun
• Tierno Jose A

Classifications

• G06N3/08

  Learning methods · CPC title

• G06N3/049 Primary

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/0499

  Feedforward networks · CPC title