Apparatus and methods of obtaining multi-scale feature vector using CNN based integrated circuits

Patent metadata

Field	Value
Publication number	US-11526723-B2
Application number	US-201916506937-A
Country	US
Kind code	B2
Filing date	Jul 9, 2019
Priority date	Jul 9, 2019
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

A pixel feature vector extraction system for extracting multi-scale features contains a cellular neural networks (CNN) based integrated circuit (IC) for extracting pixel feature vector out of input imagery data by performing convolution operations using pre-trained filter coefficients of ordered convolutional layers in a deep learning model. The ordered convolutional layers are organized in a number of groups with each group followed by a pooling layer. Each group is configured for a different size of feature map. Pixel feature vector contains a combination of feature maps from at least two groups, for example, concatenation of the feature maps. The first group of the at least two groups contains the largest size of the feature maps amongst all of the at least two groups. Feature maps of the remaining of the at least two groups are modified to match the size of the feature map of the first group.

First claim

What is claimed is:

1. A pixel feature vector extraction system comprising:
a cellular neural networks (CNN) based integrated circuit (IC) configured for extracting a pixel feature vector out of an input imagery data by performing convolution operations using pre-trained filter coefficients of ordered convolutional layers in a deep learning model, the ordered convolutional layers organized in a plurality of groups with each group followed by a pooling layer, said each group configured for a respective feature map, each feature map having a different size, the pixel feature vector containing a combination of feature maps from at least two groups, the combination concatenating respective feature maps of the at least two groups, the at least two groups including a first group and at least one remaining group, wherein the feature map of the first group has the largest size amongst the at least two groups, and wherein a respective size of the feature map of each remaining group is modified to match the size of the feature map of the first group.

2. The pixel feature vector extraction system of claim 1 , wherein the CNN based IC comprises a plurality of cellular neural networks (CNN) processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising:
a CNN processing block configured for simultaneously obtaining results of the convolution operations;
a first set of memory buffers operatively coupled to the CNN processing block for storing the input imagery data; and
a second set of memory buffers operative coupled to the CNN processing block for storing the pre-trained filter coefficients.

3. The pixel feature vector extraction system of claim 1 , wherein the respective feature maps of the at least two groups are from respective last convolutional layers of the at least two groups.

4. The pixel feature vector extraction system of claim 1 , wherein the deep learning model comprises Visual Geometry Group's VGG16 model with 13 ordered convolutional layers organized in 5 groups.

5. The pixel feature vector extraction system of claim 1 , wherein each of the pre-trained filter coefficients comprises bi-valued 3×3 filter kernel.

6. The pixel feature vector extraction system of claim 5 , wherein the pre-trained filter coefficients are obtained as a generic model.

7. The pixel feature vector extraction system of claim 1 , wherein the CNN based IC is further configured for performing activation.

8. The pixel feature vector extraction system of claim 1 , further comprises another processor for performing task specific neural network.

9. The pixel feature vector extraction system of claim 8 , wherein the task specific neural network contains at least one 3×3 convolutional layer and at least one 1×1 convolutional layer.

10. The pixel feature vector extraction system of claim 8 , wherein the task specific neural network is configured for object detection.

11. The pixel feature vector extraction system of claim 8 , wherein the task specific neural network is configured for image segmentation.

12. The pixel feature vector extraction system of claim 8 , wherein the task specific neural network is configured for optical flow application.

Assignees

• Gyrfalcon Tech Inc

Inventors

• Yang Lin
• Sun Baohua

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06V10/25 Primary

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/067

  using optical means · CPC title