Efficient image classification method based on structured pruning

Patent metadata

Field	Value
Publication number	US-11301727-B2
Application number	US-202016945626-A
Country	US
Kind code	B2
Filing date	Jul 31, 2020
Priority date	Jul 31, 2019
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

The present invention provides an efficient image classification method based on structured pruning, which incorporates a spatial pruning method based on variation regularization, including steps such as image data preprocessing, inputting images to neural network, image model pruning and retraining, and new image class predication and classification. The present invention adopts a structured pruning method that removes unimportant weight parameters of the original network model and reduces unnecessary computational and memory consumptions caused by the network model in image classification to simplify the image classifier, and then uses the sparsified network model to predict and classify new images. The simplified method according to the present invention improves the original network model in image classification efficiency by nearly two times, costs about 30% less memory consumption and produces a better classification result.

First claim

What is claimed is:

1. An efficient image classification method based on structured pruning, comprising:
(1) obtaining a training dataset and a test dataset of images; class-labeling images of the training dataset and the test dataset;
(2) preprocessing all the images of the training dataset and the test dataset;
(3) iteratively training, by using a pre-trained neural network model as an original image classifier and inputting the pre-processed images from step (2) to the neural network in batches; and pruning the original network model during the training, the pruning comprising the following steps:
(3.1) for each convolutional layer of the model, modeling a four-dimensional tensor W N*C*H*W , where N is the number of convolution kernels, C is the number of channels of the convolution kernels, and H and W are the height and width of the convolution kernel respectively;
(3.2) dividing the weights of the convolution kernels into weight groups according to the dimensions of H and W, so that weights at the same spatial position in all the channels of all the convolution kernels in a convolutional layer are grouped into an independent weight group g;
(3.3) according to a user-specified compression or speed-up ratio of the model, determining a target sparsity rate for each layer of the network model; setting a uniform regularization upper limit target_reg to all the weight groups, and assigning a regularization factor λ g to each independent weight group where all λ g are initialized to 0;
(3.4) setting a criterion to measure a sparsity importance of each weight group in the network weight matrix; sorting the weight groups in ascending order according to the criterion and obtaining an importance rank for each weight group;
(3.5) gradually assigning regularization variations Δλ g to respective weight groups, according to the importance rank of each weight group g; continuously updating the regularization factor λ g of each weight group; permanently deleting weights of a weight group when a regularization factor λ g of the weight group reaches the preset regularization upper limit target_reg;
(3.6) reshaping irregularly shaped sparse convolution kernels obtained from the spatial pruning into regular sparse convolution kernels by spatial regularization;
(3.7) deleting pixels on an input feature map in the same directions as the deleted weights on the regular sparse convolution kernels, to ensure the size of an output feature map is not changed;
(3.8) when a sparsity rate of a layer reaches the preset target sparsity rate, stopping automatically the regularization pruning of the layer; when all the layers reach their preset sparsity rates, determining completion of the pruning;

(4) retraining the sparsified network model where the deleted weights are fixed and not updated, and the other parameters are iteratively trained; when image classification accuracy of the retrained model no longer increases, stopping the retraining and outputting a sparse network model;
(5) using the sparse network model as a new image classifier, to predict class-labels of the test dataset and thus classify the new images.

2. The method according to claim 1 , wherein the preprocessing of step (2) comprises:
first, unifying the sizes of all the images to 224×224, by a resizing operation;
then, performing mean removal, to remove, for each feature to be trained, a mean value across all training images from the feature;
finally, scaling features of the images to a same range by normalization.

3. The method according to claim 1 , wherein step (3.2) comprises:
dividing into weight groups according to the two dimensions, height H and width W, of the convolution kernel of the four-dimensional tensor, so that weights (N, C, : , :) at the same spatial position in all the channels of all the convolution kernels are grouped into an independent weight group, totaling H*W weight groups.

4. The method according to claim 1 , wherein the determining a target sparsity rate for each layer of the network model in step (3.3) comprises:
when a parameter compression ratio of the model is specified, first calculating the number of parameters N*C*H*W for each parameter layer and the total number of parameters P of the model, calculating a ratio α l of the number of parameters of a layer to the total number of parameters for each layer;
then, calculating the number of parameters to be removed P′ from the model according to the preset compression ratio, calculating the number of parameters to be removed P l ′ for each layer using α l *P′, to obtain the target sparsity rate for each layer of the network model;
similarly, when a speed-up ratio of the model is specified, first calculating a floating point operations per second (GFLOPs) for each layer and a total GFLOPs of the model;
then determining a GFLOPs to be reduced for the model according to the specified speed-up ratio, calculating a GFLOPs to be reduced for each layer, to obtain the target sparsity rate for each layer.

5. The method according to claim 1 , wherein the criterion to measure a sparsity importance in step (3.4) is averaged L1−norm; and at each iteration, the weight groups are sorted in ascending order according to L1−norm, then, an average rank across the first N iterations is calculated using a function

r
¯

avg

=

1
N

⁢

∑

n
=
1

M

⁢

r
n

,

and the average ranks r avg are sorted again, to obtain a final importance rank r for each weight group.

6. The method according to claim 5 , wherein the assigning regularization variations Δλ g in step (3.5) comprises:
determining regularization variations of respective weight groups according to the importance rank r,
assigning regularization variations Δλ g to respective weight groups using the centrally-symmetric continuous function

Δ
⁢

⁢

λ
g

⁡

(
r
)

=

{

Ae

-
α

⁢

⁢
r

r
≤

Assignees

• Univ Zhejiang

Inventors

• Hu Haoji
• Li Xiang
• Wang Huan

Classifications

• G06V10/513

  Sparse representations · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454 Primary

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F18/241

  relating to the classification model, e.g. parametric or non-parametric approaches · CPC title

• G06N3/082 Primary

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title