Method and apparatus for providing random selection and long-term potentiation and depression in an artificial network

Patent metadata

Field	Value
Publication number	US-10055434-B2
Application number	US-201414513334-A
Country	US
Kind code	B2
Filing date	Oct 14, 2014
Priority date	Oct 16, 2013
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

A digital circuit element of a two dimensional dynamic adaptive neural network array (DANNA) may comprise a neuron/synapse select input functional to select the digital circuit element to function as one of a neuron and a synapse. In one embodiment of a DANNA array of such digital circuit elements, a destination neuron may be connected to a first neuron by a first synapse in one dimension, a second destination neuron may be connected to the first neuron by a second synapse in a second dimension and, optionally, a third destination neuron may be connected to the first neuron by a third synapse thus forming multiple levels of neuron and synapse digital circuit elements. In one embodiment, multiples of eight inputs may be selectively received by the digital circuit element selectively functioning as one of a neuron and a synapse. The dynamic adaptive neural network array (DANNA) may implement long-term potentiation or depression to facilitate learning through the use of an affective system and random selection of input events.

First claim

What we claim is:

1. Apparatus for a neuromorphic network comprising an artificial neural network for implementing a solution to one of a control, detection or classification application, the neuromorphic artificial neural network comprising a multi-dimensional array of addressable circuit elements configured as one of a neuron and a synapse, the neuromorphic artificial neural network comprising:
an artificial neural network configuration structure for configuring the multi-dimensional array of addressable circuit elements and an interface and control structure for connecting the two-dimensional array to an external process;
a multi-dimensional array of interconnected circuit elements, each circuit element having the same components including a long-term depression/long-term potentiation machine, an accumulator, a synapse distance/delay register and first and second neuron/synapse select leads, each circuit element addressably configured, under special purpose program control of the interface and control structure and the configuration structure, as one of a neuron and a synapse via the first neuron/synapse configuration lead to an accumulator of each circuit element of the array and a second neuron/synapse configuration lead to a synapse distance/delay register of each circuit element of the array,
at least one circuit element of the multi-dimensional array addressably representing an input neuron having a threshold,
at least one circuit element of the multi-dimensional array addressably representing an output neuron also having a threshold; the input and output neurons being connectable to an external process via the interface and control structure; and
under the special purpose program control, one to multiple circuit elements of the multi-dimensional array addressably configured as a neuron or as a synapse to addressably configure and addressably reconfigure the multi-dimensional array to solve one of the control, detection and classification applications under control of a control and optimizing device connected to the interface and control structure and the configuration structure, the addressably configured neuron or synapse connected between the input neuron and the output neuron,
the apparatus further including the long-term depression/long-term potentiation state machine of the addressably configured synapse for one of incrementing and decrementing a synaptic weight of the addressably configured synapse according to the firing of an addressably configured neuron connected to the addressably configured synapse,
the addressably configured neuron being connected to at least a selected one of a plurality of circuit elements addressably configured as a synapse to form the multi-dimensional array of circuit elements.

2. The apparatus of claim 1 , the selected circuit elements of the multi-dimensional array comprising one of a programmable logic array, an application specific integrated circuit and a very large scale integrated circuit component.

3. The apparatus of claim 2 , defined by a hardware description language, circuit diagram and/or logic equations.

4. The apparatus of claim 3 wherein said hardware description language is one of VHDL and Verilog.

5. The apparatus of claim 1 further comprising:
a storage component connected to the long-term depression/long-term potentiation state machine of the addresssably configured synapse for holding a programmable long-term depression/long-term potentiation refractory period value for the addressably configured synapse; and
wherein at least one circuit element of the apparatus comprises a digital circuit device of the addressably configured synapse, the digital circuit device comprising:
a digital multiplexer having an output port select line input for determining which circuit element of the multi-dimensional array to monitor for long-term depression/long-term potentiation.

6. The apparatus of claim 1 wherein at least one circuit element contains one of an analog electronic device and a digital electronic device, the digital electronic device comprising a counter for storing a value of a programmable long-term depression/long-term potentiation refractory period value.

7. The apparatus of claim 1 wherein a circuit element addressably configured as a neuron under the special purpose program control of the multi-dimensional array of interconnected circuit elements comprises:
an accumulator,
a comparator, responsive to the accumulator,
a programmable long-term depression/long-term potentiation refractory period input value to a counter, and
a threshold input to the comparator,
the accumulator accumulating charge from a plurality of selectable inputs of the circuit element addressably configured as the neuron for comparison at the comparator with the threshold input, and,
if the accumulated charge is greater than the threshold input and, if the long-term depression/long-term potentiation refractory period has lapsed, the comparator causing a fire output of the circuit element addressably configured as the neuron under the special purpose program control.

8. The apparatus of claim 7 further comprising:
a synapse programmable distance/delay first-in first-out register having a synapse distance/delay input, the accumulator and the synapse programmable distance/delay first-in first-out register further comprising neuron/synapse configuration input leads to each of the accumulator and the synapse programmable distance/delay first-in first-out register, the neuron/synapse configuration input leads for addressably configuring the circuit element of the multi-dimensional array as one of a neuron and a synapse under the special purpose program control.

9. The apparatus of claim 1 wherein a circuit element of the multi-dimensional array comprises;
a synapse distance/delay register having a synapse distance/delay input and a programmable first-in first-out register with sufficient entries to implement a synapse distance/delay input and to hold input fire events,
the synapse distance/delay register, responsive to an input fire multiplexer, outputting a fire signal to an element fire signal output port according to the synapse distance/delay input.

10. The apparatus of claim 1 further comprising;
an interface and control unit and a configuration unit for respectively interfacing with the multi-dimensional array of neuron and synapse circuit elements and for configuring the multi-dimensional array of neuron and synapse circuit elements under the special purpose program control of a controller, and an interface to an external process to be controlled.

11. The apparatus of claim 1 comprising:
a plurality of artificial neural networks,
one artificial neural network, during a period of off-line learning, implementing a model of an external process comprising an application of the plurality of artificial neural networks,
another artificial neural network of the plurality of artificial neural networks, during a period of on-line learning, reacting to events of the external process, and
a controller for judging performance of the at least one of the plurality of artificial neural networks from interactions with the external process, the model of the external process being modified by the controller as a result of said judged performance.

12. The apparatus of claim 1 further comprising:
a clock circuit for generating a plurality of clock signals,
the long-term depression/long-term potentiation state machine of the addressably configured synapse receiving an acquire clock input,
the clock signals of the clock circuit comprising a network clock signal input to a synapse distance/delay register and to a counter receivin

Assignees

• Univ Tennessee Res Found

Inventors

• Birdwell J Douglas
• Dean Mark E
• Schuman Catherine

Classifications

• G06N3/065

  Analogue means · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/086

  using evolutionary algorithms, e.g. genetic algorithms or genetic programming · CPC title

• G06N3/10 Primary

  Interfaces, programming languages or software development kits, e.g. for simulating neural networks · CPC title

• G06N3/02

  Neural networks · CPC title