Reconfigurable and customizable general-purpose circuits for neural networks

Patent metadata

Field	Value
Publication number	US-10628732-B2
Application number	US-201615182485-A
Country	US
Kind code	B2
Filing date	Jun 14, 2016
Priority date	Apr 8, 2011
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

A reconfigurable neural network circuit is provided. The reconfigurable neural network circuit comprises an electronic synapse array including multiple synapses interconnecting a plurality of digital electronic neurons. Each neuron comprises an integrator that integrates input spikes and generates a signal when the integrated inputs exceed a threshold. The circuit further comprises a control module for reconfiguring the synapse array. The control module comprises a global final state machine that controls timing for operation of the circuit, and a priority encoder that allows spiking neurons to sequentially access the synapse array.

First claim

What is claimed is:

1. A method comprising:
interconnecting a plurality of digital neurons via a plurality of synapses, wherein each digital neuron of the plurality of digital neurons comprises a digital counter that decays at a pre-specified decay rate for the digital neuron at each time step; and
providing noise tolerant learning for a synapse of the plurality of synapses by:
maintaining a set of bits representing a corresponding multi-bit fine-grain synaptic weight for the synapse; and
updating the corresponding multi-bit fine-grain synaptic weight for the synapse based on a learning rule and a digital counter of a digital neuron connected to the synapse.

2. The method of claim 1 , wherein, for a synapse of the plurality of synapses, a corresponding multi-bit fine-grain synaptic weight for the synapse provides noise tolerance.

3. The method of claim 1 , further comprising:
for a synapse of the plurality of synapses:
reading a corresponding multi-bit fine-grain synaptic weight for the synapse;
modifying the corresponding multi-bit fine-grain synaptic weight for the synapse based on a digital counter of a digital neuron connected to the synapse; and
writing the modified corresponding multi-bit fine-grain synaptic weight for the synapse.

4. The method of claim 1 , wherein, for a synapse of the plurality of synapses, a set of bits maintained by the synapse comprises m bits.

5. The method of claim 4 , wherein, for a synapse of the plurality of synapses, a corresponding multi-bit fine-grain synaptic weight for the synapse is a value in a range from 0 to 2 m −1.

6. The method of claim 4 , wherein, for a synapse of the plurality of synapses, the synapse has m pairs of bit lines, and an updated corresponding multi-bit fine-grain synaptic weight for the synapse is written at once utilizing only one word line.

7. A system comprising a computer processor, a computer-readable hardware storage medium, and program code embodied with the computer-readable hardware storage medium for execution by the computer processor to implement a method comprising:
interconnecting a plurality of digital neurons via a plurality of synapses, wherein each digital neuron of the plurality of digital neurons comprises a digital counter that decays at a pre-specified decay rate for the digital neuron at each time step; and
providing noise tolerant learning for a synapse of the plurality of synapses by:
maintaining a set of bits representing a corresponding multi-bit fine-grain synaptic weight for the synapse; and
updating the corresponding multi-bit fine-grain synaptic weight for the synapse based on a learning rule and a digital counter of a digital neuron connected to the synapse.

8. The system of claim 7 , wherein, for a synapse of the plurality of synapses, a corresponding multi-bit fine-grain synaptic weight for the synapse provides noise tolerance.

9. The system of claim 7 , the method further comprising:
for a synapse of the plurality of synapses:
reading a corresponding multi-bit fine-grain synaptic weight for the synapse;
modifying the corresponding multi-bit fine-grain synaptic weight for the synapse based on a digital counter of a digital neuron connected to the synapse; and
writing the modified corresponding multi-bit fine-grain synaptic weight for the synapse.

10. The system of claim 7 , wherein, for a synapse of the plurality of synapses, a set of bits maintained by the synapse comprises m bits.

11. The system of claim 10 , wherein, for a synapse of the plurality of synapses, a corresponding multi-bit fine-grain synaptic weight for the synapse is a value in a range from 0 to 2 m −1.

12. The system of claim 10 , wherein, for a synapse of the plurality of synapses, the synapse has m pairs of bit lines, and an updated corresponding multi-bit fine-grain synaptic weight for the synapse is written at once utilizing only one word line.

13. A computer program product comprising a computer-readable hardware storage medium having program code embodied therewith, the program code being executable by a computer to implement a method comprising:
interconnecting a plurality of digital neurons via a plurality of synapses, wherein each digital neuron of the plurality of digital neurons comprises a digital counter that decays at a pre-specified decay rate for the digital neuron at each time step; and
providing noise tolerant learning for a synapse of the plurality of synapses by:
maintaining a set of bits representing a corresponding multi-bit fine-grain synaptic weight for the synapse; and
updating the corresponding multi-bit fine-grain synaptic weight for the synapse based on a learning rule and a digital counter of a digital neuron connected to the synapse.

14. The computer program product of claim 13 , wherein, for a synapse of the plurality of synapses, a corresponding multi-bit fine-grain synaptic weight for the synapse provides noise tolerance.

15. The computer program product of claim 13 , the method further comprising:
for a synapse of the plurality of synapses:
reading a corresponding multi-bit fine-grain synaptic weight for the synapse;
modifying the corresponding multi-bit fine-grain synaptic weight for the synapse based on a digital counter of a digital neuron connected to the synapse; and
writing the modified corresponding multi-bit fine-grain synaptic weight for the synapse.

16. The computer program product of claim 13 , wherein, for a synapse of the plurality of synapses, a set of bits maintained by the synapse comprises m bits.

17. The computer program product of claim 16 , wherein, for a synapse of the plurality of synapses, a corresponding multi-bit fine-grain synaptic weight for the synapse is a value in a range from 0 to 2 m −1.

18. The computer program product of claim 16 , wherein, for a synapse of the plurality of synapses, the synapse has m pairs of bit lines, and an updated corresponding multi-bit fine-grain synaptic weight for the synapse is written at once utilizing only one word line.

Assignees

• IBM

Inventors

• Brezzo Bernard V
• Chang Leland
• Esser Steven K
• Friedman Daniel J
• Liu Yong
• Modha Dharmendra S
• Montoye Robert K
• Rajendran Bipin
• Seo Jae-Sun
• Tierno Jose A

Classifications

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/0499

  Feedforward networks · CPC title