Spiking neural network simulator for image and video processing

Patent metadata

Field	Value
Publication number	US-9984326-B1
Application number	US-201514680057-A
Country	US
Kind code	B1
Filing date	Apr 6, 2015
Priority date	Apr 6, 2015
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

Described is system for simulating spiking neural networks for image and video processing. The system processes an image with a spiking neural network simulator having a plurality of inter-connected modules. Each module comprises a plurality of neuron elements. Processing the image further comprises performing a neuron state update for each module, that includes aggregating input spikes and updating neuron membrane potentials, and performing spike propagation for each module, which includes transferring spikes generated in a current time step. Finally, an analysis result is output.

First claim

What is claimed is:

1. A system for simulating spiking neural networks for video and image processing, the system comprising:
one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of:
processing an image with a spiking neural network simulator having a plurality of inter-connected modules, each module comprising a plurality of neuron elements and implemented within one or more spiking neural network hardware chips, wherein processing the image further comprises:
performing a neuron state update for each module that includes aggregating input spikes and updating neuron membrane potentials; and
performing spike propagation for each module that includes transferring spikes generated in a current time step; and

outputting an analysis result,
wherein at least one buffer is associated with each module, and the spike propagation comprises updating the at least one buffer of a current module using the at least one buffer of a source module.

2. The system as set forth in claim 1 , wherein processing the image comprises:
based on a type of a task to be performed on the image, determining whether to perform the processing using one of a matrix having at least four dimensions and a matrix having two or less dimensions; and
performing the task on the image using one of the matrix having at least four dimensions and the matrix having two or less dimensions.

3. The system as set forth in claim 2 , wherein the matrix having at least four dimensions denotes synaptic connections and weights between a plurality of neuron elements of a source module and a current module in the plurality of inter-connected modules.

4. The system as set forth in claim 2 , wherein the matrix having two or less dimensions is configured to perform a convolution.

5. The system as set forth in claim 3 , wherein processing the image further comprises changing the weights for the matrix having at least four dimensions of at least one of the modules when learning occurs.

6. The system as set forth in claim 1 , wherein the spike propagation steps between modules located on the same computer environment are performed by assigning pointers.

7. A computer-implemented method for simulating spiking neural networks for video and image processing, comprising an act of:
causing a data processor to execute instructions stored on a non-transitory memory such that upon execution, the data processor performs operations of:
processing an image with a spiking neural network simulator having a plurality of inter-connected modules, each module comprising a plurality of neuron elements and implemented within one or more spiking neural network hardware chips, wherein processing the image further comprises:
performing a neuron state update for each module that includes aggregating input spikes and updating neuron membrane potentials; and
performing spike propagation for each module that includes transferring spikes generated in a current time step; and

outputting an analysis result,
wherein at least one buffer is associated with each module, and the spike propagation comprises updating the at least one buffer of a current module using the at least one buffer of a source module.

8. The method as set forth in claim 7 , wherein processing the image comprises:
based on a type of a task to be performed on the image, determining whether to perform the processing using one of a matrix having at least four dimensions and a matrix having two or less dimensions; and
performing the task on the image using one of the matrix having at least four dimensions and the matrix having two or less dimensions.

9. The method as set forth in claim 8 , wherein the matrix having at least four dimensions denotes synaptic connections and weights between a plurality of neuron elements of a source module and a current module in the plurality of inter-connected modules.

10. The method as set forth in claim 8 , wherein the matrix having two or less dimensions is configured to perform a convolution.

11. The method as set forth in claim 9 , wherein processing the image further comprises changing the weights for the matrix having at least four dimensions of at least one of the modules when learning occurs.

12. The method as set forth in claim 7 , wherein the spike propagation steps between modules located on the same computer environment are performed by assigning pointers.

13. A computer program product for simulating spiking neural networks for video and image processing, the computer program product comprising:
computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:
processing an image with a spiking neural network simulator having a plurality of inter-connected modules, each module comprising a plurality of neuron elements and implemented within one or more spiking neural network hardware chips, wherein processing the image further comprises:
performing a neuron state update for each module that includes aggregating input spikes and updating neuron membrane potentials; and
performing spike propagation for each module that includes transferring spikes generated in a current time step; and

outputting an analysis result,
wherein at least one buffer is associated with each module, and the spike propagation comprises updating the at least one buffer of a current module using the at least one buffer of a source module.

14. The computer program product as set forth in claim 13 , wherein processing the image comprises:
based on a type of a task to be performed on the image, determining whether to perform the processing using one of a matrix having at least four dimensions and a matrix having two or less dimensions; and
performing the task on the image using one of the matrix having at least four dimensions and the matrix having two or less dimensions.

15. The computer program product as set forth in claim 14 , wherein the matrix having at least four dimensions denotes synaptic connections and weights between a plurality of neuron elements of a source module and a current module in the plurality of inter-connected modules.

16. The computer program product as set forth in claim 14 , wherein the matrix having two or less dimensions is configured to perform a convolution.

17. The computer program product as set forth in claim 15 , wherein processing the image further comprises changing the weights for the matrix having at least four dimensions of at least one of the modules when learning occurs.

18. The computer program product as set forth in claim 13 , wherein the spike propagation steps between modules located on the same computer environment are performed by assigning pointers.

19. The system as set forth in claim 1 , wherein each module comprises an input layer, one or more middle layers, and an output layer.

20. The method as set forth in claim 7 , wherein each module comprises an input layer, one or more middle layers, and an output layer.

21. The method as set forth in claim 13 , wherein each module comprises an input layer, one or more middle layers, and an output layer.

Assignees

• Hrl Lab Llc

Inventors

• Chen Yang
• Cao Yongqiang
• Khosla Deepak

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06K9/62

  Physics · mapped topic

• G06N3/10 Primary

  Interfaces, programming languages or software development kits, e.g. for simulating neural networks · CPC title

• G06N3/08

  Learning methods · CPC title