Method and apparatus for constructing a dynamic adaptive neural network array (DANNA)

Patent metadata

Field	Value
Publication number	US-10095718-B2
Application number	US-201414513297-A
Country	US
Kind code	B2
Filing date	Oct 14, 2014
Priority date	Oct 16, 2013
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

A circuit element of a multi-dimensional dynamic adaptive neural network array (DANNA) may comprise a neuron/synapse select input functional to select the circuit element to function as one of a neuron and a synapse. In one embodiment of a DANNA array of such circuit elements, (wherein a circuit element or component thereof may be analog or digital), a destination neuron may be connected to a first neuron by a first synapse in one dimension, a second destination neuron may be connected to the first neuron by a second synapse in a second dimension and, optionally, a third destination neuron may be connected to the first neuron by a third synapse. The DANNA may thus form multiple levels of neuron and synapse circuit elements. In one embodiment, multiples of eight inputs may be selectively received by the circuit element selectively functioning as one of a neuron and a synapse. The dynamic adaptive neural network array (DANNA) may comprise a special purpose processor for performing one of a control, anomaly detection and classification application and may comprise a first structure connected to a neuroscience-inspired dynamic artificial neural network (NIDA), comprise substructures thereof or be combined with other neural networks.

First claim

What we claim is:

1. Apparatus for a neuromorphic network comprising an artificial neural network for implementing a solution to one of a control, detection and classification application, the neuromorphic artificial neural network comprising a multi-dimensional array of interconnected, addressable circuit elements configured as one of a neuron and a synapse, the neuromorphic artificial neural network comprising:
a special purpose computer processor comprising an artificial neural network configuration structure for configuring the multi-dimensional array of interconnected, addressable circuit elements; and
an interface and control structure for connecting the addressably configured multi-dimensional array to an external process implementing one of a control, detection and classification application;
each circuit element having the same electronic components addressably configured to perform, under special purpose program control by of the special purpose computer processor, one of a neuron function and a synapse function each addressably configured circuit element of the addressably configured multi-dimensional array,
at least one circuit element of the multi-dimensional array addressably configured as an input neuron responsive to receiving data from the external process,
at least one circuit element of the addressably configured multi-dimensional array addressably configured as an output neuron for outputting data to the external process, and
under special purpose program control, one to multiple circuit elements of the multi-dimensional array addressably configured as one of a neuron or a synapse connected between the input neuron and the output neuron,
each addressably configured synapse of the neuromorphic network of the addressably configured multi-dimensional array of circuit elements having a programmable propagation distance/delay parameter,
each addressably configured neuron addressably connected under special purpose program control to one of up to 8N different addressably configured synapses of the addressably configured multi-dimensional array, where N is an integer equal to or greater than 1.

2. The apparatus of claim 1 , the selected elements of the multi-dimensional array comprising a programmable logic array, ASIC or VLSI component.

3. The apparatus of claim 2 , defined by a hardware description language, circuit diagram and/or logic equations.

4. The apparatus of claim 3 wherein said hardware description language is one of VHDL and Verilog.

5. The apparatus of claim 1 wherein at least one circuit element is a digital circuit element comprising a register electronic component for storing a digital value of the programmable synapse distance/delay parameter,
the synapse distance/delay register electronic component having a neuron/synapse addressable select lead serving as a synapse distance/delay input lead if the circuit element of the addressably configured multi-dimensional array has been addressably configured to perform a synapse function, the neuron/synapse addressable select lead for addressably inputting a selection of the at least one circuit element as functioning as one of a neuron and a synapse of the addrcssably configured multi-dimensional array by the special purpose program processor,
the synapse distance/delay register electronic component further comprising sufficient entries to store a digital value of the synapse distance/delay parameter and to hold at least one fire event.

6. The apparatus of claim 1 wherein at least one addressably configured circuit element of the addrcssably configured multi-dimensional neural network array contains an analog electronic device.

7. The apparatus of claim 6 wherein said analog electronic device is an analog signal storage device.

8. The apparatus of claim 1 , the special purpose program control implemented using one of a microprocessor, a microcontroller, and a processor core.

9. The apparatus of claim 1 wherein the multi-dimensional array comprises a minimum of an input neuron, a first synapse, a first neuron, and an output neuron, a given circuit element addressably configured as a first neuron is adapted to be connected to at least one addressably configured output neuron via at least one addressably configured first or second synapse,
the given circuit element, when addressably configured as a neuron, comprising an accumulator electronic component, responsive to a threshold input and a multiplexer electronic component, for accumulating charge values until the threshold is reached and, when the given circuit element is addressably and selectively configured as a synapse, the accumulator electronic component for receiving a signal for one of incrementing and decrementing synaptic weight,
and a comparator electronic component for comparing the threshold with the accumulated charge value, the comparator electronic component actuating the first neuron circuit element to fire if the first neuron circuit element is not in its refractory period, the fire event being held in a synapse distance/delay register electronic component, and
each connection between the connected, addressably configured first neuron circuit element connected to at least one of the first and second addressably configured synapse circuit elements, the connections resulting in an at least one level addressably configured multi-dimensional neural network structure.

10. The apparatus of claim 1 wherein a circuit element selectively representing and addressably configured as a neuron of the multi-dimensional array receives up to eight inputs in multiples of eight inputs from addressably configured and connected first and second selectively represented synapses.

11. The apparatus of claim 1 wherein a circuit element of the multi-dimensional array of interconnected circuit elements addressably configured as a neuron comprises an accumulator electronic component,
a comparator electronic component responsive to the accumulator electronic component,
a refractory period input to a counter electronic component and
a threshold input to the accumulator electronic component, the accumulator electronic component accumulating charge from a plurality of selectable inputs of the circuit element addressably configured as a neuron for comparison at the comparator electronic component with the threshold input, and, if the accumulated charge is greater than the threshold input and, if a refractory period has lapsed, the comparator electronic component causing a fire output of the circuit element addressably configured as a neuron, where the accumulator electronic component is shared between the neuron function and the synapse function of the addressably configured circuit element.

12. The apparatus of claim 1 wherein a circuit element of the multi-dimensional array, when addressably configured as a synapse, comprises:
a synapse distance/delay register electronic component having a synapse distance/delay input serving as a neuron/synapse select lead for addressably configuring the circuit element as a synapse and comprises
a first-in, first-out register with sufficient entries to simultaneously represent a digital value of the synapse distance/delay and to hold multiple input fire events,
the synapse distance/delay register electronic component, responsive to an input fire multiplexer electronic component, outputting a fire signal to an element fire signal line according to the synapse distance/delay input.

13. The apparatus of claim 1 wherein a plurality of circuit elements addressably configured as one of neurons performing a neuron function and synapses performing a synapse function,
a plurality of circuit elements o

Assignees

• Univ Tennessee Res Found

Inventors

• Birdwell J Douglas
• Dean Mark E
• Schuman Catherine

Classifications

• G06N3/065

  Analogue means · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/10 Primary

  Interfaces, programming languages or software development kits, e.g. for simulating neural networks · CPC title

• G06N3/02

  Neural networks · CPC title

• G06N3/086

  using evolutionary algorithms, e.g. genetic algorithms or genetic programming · CPC title