In-memory spiking neural networks for memory array architectures

Patent metadata

Field	Value
Publication number	US-11354568-B2
Application number	US-201715639997-A
Country	US
Kind code	B2
Filing date	Jun 30, 2017
Priority date	Jun 30, 2017
Publication date	Jun 7, 2022
Grant date	Jun 7, 2022

Abstract

Systems, apparatuses and methods may provide for a chip that includes a memory array having a plurality of rows corresponding to neurons in a spiking neural network (SNN) and a row decoder coupled to the memory array, wherein the row decoder activates a row in the memory array in response to a pre-synaptic spike in a neuron associated with the row. Additionally, the chip may include a sense amplifier coupled to the memory array, wherein the sense amplifier determines post-synaptic information corresponding to the activated row. In one example, the chip includes a processor to determine a state of a plurality of neurons in the SNN based at least in part on the post-synaptic information and conduct a memory array update, via the sense amplifier, of one or more synaptic weights in the memory array based on the state of the plurality of neurons.

First claim

We claim:

1. A chip comprising:
a memory array including a plurality of rows corresponding to neurons in a spiking neural network (SNN);
a row decoder coupled to the memory array, the row decoder to activate a row in the tnetnory array in response to a pre-synaptic spike in a neuron associated with the row;
a sense amplifier coupled to the memory array, the sense amplifier to determine post-synaptic information corresponding to the activated row; and
a processor including logic coupled to the sense amplifier, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic coupled to the sense amplifier to:
determine a state of a plurality of neurons in the SNN based on the post-synaptic information; and
conduct a memory array update, via the sense amplifier, of one or more synaptic weights in the memory array based on the state of the plurality of neurons and a learning heuristic, wherein the memory array update is to bypass off-chip communications by operations to restore the values from the sense amplifier to the memory array, wherein the memory array update is to include an addition of an entry to a local spike queue, and wherein the local spike queue is populated by one or more remote memory arrays via a remote spike queue,
wherein a plurality of synapses in the plurality of rows of the memory array are grouped according to a synaptic delay and the synaptic weight associated with each synapse.

2. The chip of claim 1 , further including the local spike queue, wherein the local spike queue is coupled to the processor and the row decoder, and wherein the entry is to include information about a pre-synaptic neuron.

3. The chip of claim 1 , further including the remote spike queue, wherein the remote spike queue is coupled to the one or more remote memory arrays, wherein the memory array update is to include an addition of an entry to the remote spike queue, and wherein the entry is to include information about a pre-synaptic neuron.

4. The chip of claim 1 , wherein the post-synaptic information is to include one or more of weight information, target neuron identifier information, delay information, plasticity information or type information, wherein the state of the plurality of neurons is to be maintained on the processor, and wherein the state is to include one or more of membrane potential, refractory state, recent spike timing or learning metadata.

5. A method of operating a chip, comprising:
activating, by a row decoder coupled to a memory array including a plurality of rows corresponding to neurons in a spiking neural network (SNN), a row in the memory array in response to a pre-synaptic spike in a neuron associated with the row;
determining, by a sense amplifier coupled to the memory array, post-synaptic information corresponding to the activated row;
determining, by a processor includinu logic coupled to the sense am ate of a plurality of neurons in the SNN based on the post-synaptic information; and
conducting a memory array update, by the sense amplifier, of one or more synaptic weights in the memory array, wherein the memory array update is to bypass off-chip communications by operations to restore the values from the sense amplifier to the memory array, wherein the memory array update is to include an addition of an entry to a local spike queue, and wherein the local spike queue is populated by one or more remote memory arrays via a remote spike queue,
wherein a plurality of synapses in the plurality of rows of the memory array are grouped according to a synaptic delay and the synaptic weight associated with each synapse.

6. The method of claim 5 , further comprising:
wherein the memory array update is based on the state of the plurality of neurons.

7. The method of claim 5 , wherein the memoryarray update is conducted further based on a learning heuristic.

8. The method of claim 5 , wherein the entry includes information about a pre-synaptic neuron.

9. The method of claim 5 , wherein the post-synaptic information includes one or more of weight information, target neuron identifier information, delay information, plasticity information or type information.

10. A system comprising:
a power supply; and
a chip, the chip comprising:
a memory array including a pluralitv of rows corresponding to neurons in a spiking neural network (SNN);
a row decoder coupled to the memory array, the row decoder to activate a row in the memory array in response to a pre-synaptic spike in a neuron associated with the row;
a sense amplifier coupled to the memory array, the sense amplifier to determine tic information corresponding to the activated row and
a processor including logic coupled to the sense amplifier, wherein the logic is implemented at least partly in one or more of configurable logic is or fixed-functional hardware logic, the logic coupled to the sense amplifier to:
determine a state of a plurality of neurons in the SNN based on the post-synaptic information; and
conduct a memory array update, via the sense amplifier, of one or more synaptic weights in the memory array based on the state of the plurality of neurons and a learning heuristic, wherein the memory array update is to bypass off-chip communications by operations to restore the values from the sense amplifier to the memory array, wherein the memo arra a date is to include an addition of an entry to a local spike queue, and wherein the local spike queue is populated by one or more remote memory arrays via a remote spike queue,

wherein a plurality of synapses in the plurality of rows of the memory array are grouped according to a synaptic delay and the synaptic weight associated with each synapse.

11. The chip of claim 10 , wherein the memory array update is to be conducted further based on a learning heuristic.

12. The chip of claim 10 , further including the local spike queue, wherein the local spike queue is coupled to the processor and the row decoder, and wherein the entry is to include information about a pre-synaptic neuron.

13. The chip of claim 10 , further including the remote spike queue, wherein the remote spike queue is coupled to the one or more remote memory arrays, wherein the memory array update is to include an addition of an entry to the remote spike queue, and wherein the entry is to include information about a pre-synaptic neuron.

14. The chip of claim 10 , wherein the post-synaptic information is to include one or more of weight information, target neuron identifier information, delay information, plasticity information or type information.

15. At least one non-transitory computer readable storage medium comprising a set of instructions, which when executed by a chip, cause the chip to:
activate a row in a memory array including a plurality of rows corresponding to neurons in a spiking neural network (SNN) in response to a pre-synaptic spike in a neuron associated with the row,
determine, by a sense amplifier coupled to the memory array, post-synaptic information corresponding to the activated row;
determine a state of a plurality of neurons in the spiking neural network (SNN) based on the post-synaptic information from the sense amplifier on the chip, wherein the post-synaptic information is to correspond to the activated row in the plurality of rows of the memory array on the chip; and
conduct a memory array update, via the sense amplifier, of one or more synaptic weights in the memory array based on the state of the plurality of neurons, wherein the memory array update is to bypass off-chip communications by operations to restore the value

Assignees

• Intel Corp

Inventors

• Akin Berkin
• Pugsley Seth H

Classifications

• G06N3/08

  Learning methods · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/049 Primary

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N20/00

  Machine learning · CPC title