Dynamically reconstructable multistage parallel single instruction multiple data array processing system

Patent metadata

Field	Value
Publication number	US-9449257-B2
Application number	US-201214649859-A
Country	US
Kind code	B2
Filing date	Dec 4, 2012
Priority date	Dec 4, 2012
Publication date	Sep 20, 2016
Grant date	Sep 20, 2016

Abstract

The present invention proposes a dynamically reconfigurable multi-level parallel single instruction multiple data array processing system which has a pixel level parallel image processing element array and a row-parallel array processor. The PE array mainly implements a linear operation which is adapted to be executed in parallel in the low and middle levels of image processing and the RP array implements an operation which is adapted to execute in row-parallel in the low and middle levels of image processing or more complex nonlinear operations. In particularly, such a system can be dynamically reconfigured as an SOM neural network at a low cost of area, and the neural network supports high level of image processing such as a high speed online neural network training and image feature recognition, and completely overcomes a defect that a high level of image processing can't be done by pixel-level parallel processing array in the existing programmable vision chips and parallel vision processors, and facilitates an intelligent and portable real time on-chip vision image system with a complete function at low device cost and low power consumption.

First claim

We claim:

1. A dynamically reconfigurable multi-level parallel single instruction multiple data array processing system, which is applied for rapid image feature extraction and feature recognition of high speed vision image on a high speed on-chip vision system, the system comprising:
a configurable image sensor interface ( 1 ) used to receive pixel data from an image sensor serially or in parallel and to output the pixel data in row-parallel to a subsequent dynamically reconfigurable parallel image processing element array and self organizing map (SOM) neural network ( 2 );
the dynamically reconfigurable parallel image processing element array and self organizing map (SOM) neural network ( 2 ) with dynamical reconfigurability to be an M×M pixel-level parallel processing element (PE) array or an (M/4)×(M/4) self organizing map SOM neural network so as to implement image feature extraction and image feature recognition during the respective stages of the image processing, wherein M is a natural number;
a row processor array ( 5 ) used to assist the dynamically reconfigurable parallel image processing element array and self organizing map (SOM) neural network ( 2 ) to implement relative image processing tasks during the respective stages of image processing adapted to being parallel processed by row, to implement a rapid nonlinear processing and a global image processing, and to serially shift in data and output processing results under external control of the system; and
an array controller ( 8 ) used to, under control of a driver external to the system, read control instructions for controlling the dynamically reconfigurable parallel image processing element array and self organizing map (SOM) neural network ( 2 ) and the RP array ( 5 ) from a variable very long instruction word single instruction multiple data VVS instruction memory internal to the system, and used to decode the control instructions along with some values of their own specific registers to be output to the dynamically reconfigurable parallel image processing element array and self organizing map (SOM) neural network ( 2 ) and the RP array ( 5 ) as an array control signal.

2. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 1 , wherein dynamically reconfigurable parallel image processing element array and self organizing map (SOM) neural network ( 2 ) comprises M×M of fine-grained parallel image processing element PE ( 3 ) which are operated in a single instruction multiple data SIMD mode in a pixel-level parallel fashion.

3. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 2 , every block of 4×4 PE sub-array ( 4 ) in the dynamically reconfigurable parallel image processing element array and self organizing map (SOM) neural network ( 2 ) is able to be dynamically reconfigured to be one SOM neuron in the SOM neural network.

4. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 3 , wherein before the 4×4 PE sub-array 4 is reconfigured into an SOM neuron, every PE unit ( 11 ) is connected to its four nearest PE units and communicates data with them in a bandwidth of 1 bit.

5. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 4 , wherein the PE unit on boundaries of the 4×4 PE sub-array 4 is connected to the PE unit of the adjacent 4×4 PE sub-array 4 and communicates data with them.

6. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 4 , wherein every PE unit comprises a one bit arithmetic logic unit ALU ( 13 ), one carry register Creg ( 12 ) and a local memory ( 14 ) with a bit width of 1 bit and a bit depth of W, wherein W is a natural number; operands of the arithmetic logic unit ( 13 ) are from memories in its own PE unit or the adjacent PE unit; the carry-out generated during the operation are stored into the carry register in the PE unit and functions as a carry-in of the arithmetic logic unit for the next operation, so as to implement a multiple bit operation in such a bit-serial fashion.

7. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 3 , wherein the 4×4 PE sub-array ( 4 ) is reconfigured to be one neuron ( 15 ) in the SOM neural network, in which topological connections of the PE units changes, and each PE unit acts as one “bit-slice” of the reconfigured SOM neuron; every PE units contribute corresponding operation sources and memory sources to some bit in the neuron where the PE unit is located, so it is mapped to be some bit in the SOM neuron; at this moment, the PE unit is only connected to two PE units which are mapped to the adjacent bits and communicated data with them, and the band width is increased to be two bit including one bit of memory data communication and one bit of carry data communication.

8. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 7 , wherein every carry-out of the PE unit functioning as the “bit slice” is not stored into its own carry register, but directly functions as a carry-in of the arithmetic logic unit in the adjacent “bit slice” PE at higher bit position among the reconfigured SOM neurons; the arithmetic logic units for all of the sixteen PE units are connected together to form one 16-bit arithmetic logic unit, and the final carry-out of such a 16-bit arithmetic logic unit is saved into a sign indicator register ( 17 ) of the neuron and functions as a carry-in or a conditional operation indicator for the subsequent operations.

9. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 8 , wherein the respective bit operands of the 16-bit arithmetic logic unit are simultaneously from the memories of all the PE units, so the neuron ( 15 ) corresponds to a local memory ( 20 ) with a bit width of 16-bits and a bit depth of W.

10. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 9 , wherein the neuron ( 15 ) obtained by reconstructing the 4×4 PE sub-array ( 4 ) is capable of communicating data with the its left and right neighboring neurons in a communication bandwidth of 1-bit; and the SOM neuron ( 15 ) comprises one 16-bits ALU ( 18 ), one shift control unit ( 19 ), a second local memory ( 20 ) with a bit width of 16-bits and a bit depth of W, 16-bit temporal register as well as some additional flag registers.

11. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 1 , wherein the particular circuit arrangement of the PE unit comprises one 1-bit ALU unit ( 23 ), a first operand selector ( 26 ), a second operand selector ( 25 ), a local dual-port memory ( 28 ) with a bit width of 1-bit and a bit depth of W, one condition selector ( 29 ), one carry register ( 22 ), one temporary register ( 24 ) and some multiplexers relevant to operand selection and dynamical reconfiguration, and the two inputs of these multiplexers relevant to reconfiguration are marked as PE and SOM which represent to implement a data selection function relevant to reconfiguration.

12. The dynamically reconfigurable multi-level parallel single instruction multiple data array processing system according to claim 11 , wherein in order to reconfigure into a s

Assignees

• Inst Semiconductors Cas

Inventors

• Shi Cong
• Wu Nanjian
• Long Xitian
• Yang Jie
• Qin Qi

Classifications

• G06F15/8023 Primary

  Two dimensional arrays, e.g. mesh, torus · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06F18/2137

  based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps · CPC title

• G06N3/045

  Combinations of networks · CPC title