System for implementing a sparse coding algorithm

Patent metadata

Field	Value
Publication number	US-2016358075-A1
Application number	US-201615176910-A
Country	US
Kind code	A1
Filing date	Jun 8, 2016
Priority date	Jun 8, 2015
Publication date	Dec 8, 2016
Grant date	—

Abstract

A sparse coding system. The sparse coding system comprises a neural network including a plurality of neurons each having a respective feature associated therewith and each being configured to be electrically connected to every other neuron in the network and to a portion of an input dataset. The plurality of neurons are arranged in a plurality of neuron clusters each comprising a respective subset of the plurality of neurons, and the neurons in each cluster are electrically connected to one another in a bus structure, and the plurality of clusters are electrically connected together in a ring structure. Also provided is a sparse coding system that comprises an inference module configured to extract features from an input image containing an object, wherein the inference module comprises an implementation of a sparse coding algorithm, and a classifier configured to classify the object in the input image based on the extracted features.

First claim

1 . A sparse coding system, comprising:
a neural network including a plurality of neurons each having a respective feature associated therewith and each being configured to be electrically connected to every other neuron in the network and to a portion of an input dataset, wherein the plurality of neurons are arranged in a plurality of neuron clusters each comprising a respective subset of the plurality of neurons, and further wherein the neurons in each cluster are electrically connected to one another in a bus structure, and the plurality of clusters are electrically connected together in a ring structure.

2 . The system of claim 1 , wherein the bus structure is a multi-dimensional bus structure comprising a plurality of rows, a plurality of columns, and a plurality of logic OR gates, wherein each OR gate is associated with a respective row or column and electrically connects the neurons in that row or column to one another.

3 . The system of claim 1 , wherein the bus structure is a multi-dimensional bus structure having A rows and B columns of neurons, and further wherein the bus structure comprises A horizontal buses each connecting B neurons in a respective row of the bus structure, and B vertical buses each connecting A neurons in a respective column of the bus structure.

4 . The system of claim 1 , further comprising a memory, and wherein each connection between two neurons has a respective weight W associated therewith and each connection between a neuron and at least a portion of the input dataset has a respective weight Q associated therewith, and further wherein each weight Q and W is stored in the memory of the system.

5 . The system of claim 4 , wherein the weights Q and W are quantized to a fixed-point number to reduce memory storage.

6 . The system of claim 4 , wherein the memory is partitioned into a first portion and a second portion, and further wherein both the first and second portions are used during a learning operation performed by the system, and only one of the first and second portions is used during an inference operation performed by the system.

7 . The system of claim 6 , wherein the first portion of the memory comprises the most significant bits (MSBs) of the Q and W weights, and the second portion of the memory comprises the least significant bits (LSBs) of the Q and W weights.

8 . The system of claim 1 , wherein the neural network is configured to perform a learning operation based on a first batch of neurons to fire.

9 . The system of claim 1 , wherein the neural network is configured to perform a learning operation, and wherein parameter updates during the learning operation are carried out using a message passing approach.

10 . The system of claim 1 , wherein each neuron is configured to generate a binary spike output.

11 . The system of claim 1 , the neural network comprises an inference module configured to extract features from an image represented by the input dataset, wherein the image contains an object.

12 . The system of claim 11 , further comprising a classifier configured to classify the object in the input image based on the extracted features.

13 . The system of claim 1 , wherein the bus structure comprises an arbitration-free bus structure.

14 . The system of claim 1 , further comprising a power supply, and wherein a supply voltage supplied by the power supply to the neural network is scaled to take advantage of the error resilience of the sparse coding system.

15 . A sparse coding system, comprising:
an inference module configured to extract features from an input image containing an object, wherein the inference module comprises an implementation of a sparse coding algorithm; and a classifier configured to classify the object in the input image based on the extracted features, wherein the inference module and classifier are integrated on a single chip.

16 . The system of claim 15 , wherein the inference module comprises at least one neural network comprising a plurality of neurons each having a respective feature associated therewith and each being configured to be connected to every other neuron in the network and at least a portion of the input image.

17 . The system of claim 16 , wherein the at least one neural network has a scalable multi-layer architecture.

18 . The system of claim 17 , wherein the at least one neural network comprises a plurality of neuron clusters each comprising a respective subset of the plurality of neurons, and further wherein the neurons in each cluster are electrically connected to one another in a bus structure, and the plurality of clusters are electrically connected together in a ring structure.

19 . The system of claim 18 , wherein the bus structure is a multi-dimensional bus structure comprising a plurality of rows, a plurality of columns, and a plurality of logic OR gates, wherein each OR gate is associated with a respective row or column and electrically connects the neurons in that row or column to one another.

20 . The system of claim 16 , wherein the inference module further comprises a memory, and wherein each connection between two neurons in the neural network has a respective weight W associated therewith and each connection between a neuron in the neural network and at least a portion of the input image has a respective weight Q associated therewith, and further wherein each weight W and Q is stored in the memory.

21 . The system of claim 20 , wherein the memory is partitioned into a first portion and a second portion, and further wherein both the first and second portions are used during a learning operation performed by the inference module and only one of the first and second portions is used during an inference operation performed by the inference module.

22 . The system of claim 15 , wherein the classifier comprises an event-driven implementation of a classifier.

23 . The system of claim 15 , wherein the classifier comprises one or more adders and does not comprise any multipliers.

24 . An object recognition system comprising the system of claim 15 , wherein the inference module comprises a front-end of the object recognition system and the classifier comprises a back-end of the object recognition system.

Assignees

• Univ Michigan Regents

Inventors

• Zhang Zhengya
• Chen Thomas
• Kim Jung Kuk
• Knag Phil

Classifications

• G06N3/063

  using electronic means · CPC title

• G06N3/088 Primary

  Non-supervised learning, e.g. competitive learning · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title