Approximating fully-connected layers with multiple arrays of 3x3 convolutional filter kernels in a CNN based integrated circuit

Patent metadata

Field	Value
Publication number	US-10366328-B2
Application number	US-201815920842-A
Country	US
Kind code	B2
Filing date	Mar 14, 2018
Priority date	Sep 19, 2017
Publication date	Jul 30, 2019
Grant date	Jul 30, 2019

Abstract

Multiple 3×3 convolutional filter kernels are used for approximating operations of fully-connected (FC) layers. Image classification task is entirely performed within a CNN based integrated circuit. Output at the end of ordered convolutional layers contains P feature maps with F×F pixels of data per feature map. 3×3 filter kernels comprises L layers with each organized in an array of R×Q of 3×3 filter kernels, Q and R are respective numbers of input and output feature maps of a particular layer of the L layers. Each input feature map of the particular layer comprises F×F pixels of data with one-pixel padding added around its perimeter. Each output feature map of the particular layer comprises (F−2)×(F−2) pixels of useful data. Output of the last layer of the L layers contains Z classes. L equals to (F−1)/2 if F is an odd number. P, F, Q, R and Z are positive integers.

First claim

What is claimed is:

1. A digital integrated circuit comprising: a plurality of cellular neural networks (CNN) processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising: a CNN processing block configured for simultaneously performing convolutional operations using input data and pre-trained filter coefficients of a plurality of ordered convolutional layers, and further configured for classifying the input data using a plurality of 3×3 filter kernels to approximate operations of fully-connected (FC) layers, wherein output of the plurality of ordered convolutional layers has P feature maps with F×F pixels of data per feature map and the plurality of 3×3 filter kernels comprises L layers with each of the L layers organized in an array of R×Q of 3×3 filter kernels, wherein Q and R are respective numbers of input and output feature maps of a particular layer of the L layers, wherein L is equal to (F−1)/2 when F is an odd number, and wherein P, F, Q and R are positive integers; a first set of memory buffers operatively coupling to the CNN processing block for storing the input data; and a second set of memory buffers operative coupling to the CNN processing block for storing the pre-trained filter coefficients.

2. The digital integrated circuit of claim 1 , wherein the array of R×Q of 3×3 filter kernels obtains results of convolutional operations of the particular layer.

3. The digital integrated circuit of claim 2 , when each input feature map of the particular layer comprises F×F pixels of data with one-pixel padding added around perimeter, each output feature map of the particular layer comprises (F−2)×(F−2) pixels of useful data located in center portion of said each output feature map.

4. The digital integrated circuit of claim 3 , wherein output of the last layer of the L layers contains Z classes for image classification, each of the Z classes contains 1×1 pixel of useful data stored in center pixel of the F×F pixels, where Z is a positive integer.

5. The digital integrated circuit of claim 3 , wherein output of the last layer of the L layers contains S nodal feature values, each of the S nodal feature values contains 1×1 pixel of useful data stored in center pixel of the F×F pixels, where S is a positive integer.

6. The digital integrated circuit of claim 5 , wherein the plurality of 3×3 filter kernels further includes at least one inner-product replacement array of U×T of 3×3 filter kernels used for replacing inner-product operations in one of the FC layers that contains T number of input nodal feature values and U number of output nodal feature values, where T and U are positive integers.

7. The digital integrated circuit of claim 6 , wherein FC layer weights of the innerproduct operations are respectively stored in center pixel of each 3×3 filter kernel in the at least one inner-product replacement array of U×T of 3×3 filter kernels.

8. The digital integrated circuit of claim 7 , wherein numerical zeros are stored in eight perimeter pixels of said each 3×3 filter kernel in the at least one inner-product replacement array of U×T of 3×3 filter kernels.

9. The digital integrated circuit of claim 2 , when each input feature map of the particular layer comprises F×F pixels of data without padding, each output feature map of the particular layer comprises (F−2)×(F−2) pixels.

10. The digital integrated circuit of claim 1 , wherein the CNN processing block is further configured for performing operations of activation and pooling.

Assignees

• Gyrfalcon Tech Inc

Inventors

• Yang Lin
• Dong Patrick Z
• Dong Jason Z
• Sun Baohua

Classifications

• G06N3/048

  Activation functions · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title