Device and method for improving processing speed of neural network and application thereof

Patent metadata

Field	Value
Publication number	US-2018189650-A1
Application number	US-201715799151-A
Country	US
Kind code	A1
Filing date	Oct 31, 2017
Priority date	Dec 30, 2016
Publication date	Jul 5, 2018
Grant date	—

Abstract

A device and a method for improving a processing speed of a neural network and applications thereof in the neural network where the device includes a processor configured to perform: determining, according to a predetermined processing speed improvement target, a dimension reduction amount of each of one or more parameter matrixes in the neural network obtained through training; preprocessing each parameter matrix based on the dimension reduction amount of the parameter matrix; and retraining the neural network based on a result of the preprocessing to obtain one or more dimension reduced parameter matrixes so as to ensure performance of the neural network meets a predetermined requirement. According to the embodiments of the present disclosure, it is possible to significantly improve the processing speed of the neural network while ensuring the performance of the neural network meets the predetermined requirement.

First claim

What is claimed is:

1 . A device for improving processing speed of a neural network, the device comprising a processor configured to perform:
determining, according to a predetermined processing speed improvement target, a dimension reduction amount of each of at least one parameter matrix in the neural network obtained through training; preprocessing each parameter matrix based on the dimension reduction amount of the parameter matrix; and retraining the neural network based on a result of the preprocessing to obtain at least one dimension reduced parameter matrix to ensure performance of the neural network meets a predetermined requirement.

2 . The device according to claim 1 , wherein the dimension reduction amount represents a columns dimension reduction amount of each parameter matrix, and the processor is further configured to perform the pre-processing by performing operations for each parameter matrix comprising:
calculating a column score of each of the columns of the parameter matrix according to values of parameters in each column of the parameter matrix; and zeroing, according to the column dimension reduction amount of the parameter matrix, the parameters in a column where the column score meets a predetermined condition.

3 . The device according to claim 2 , wherein the processor is further configured to calculate, for each parameter matrix, a sum of absolute values of the parameters in each column of the parameter matrix as the column score of the column.

4 . The device according to claim 2 , wherein the processor is further configured to calculate, for each parameter matrix, the column score according to loss weights associated with the parameters in each column of the parameter matrix.

5 . The device according to claim 4 , wherein the processor is further configured to:
normalize all of the parameters and the loss weights in each parameter matrix; and calculate, for each parameter matrix, a sum of weighted sums of normalized parameters and normalized loss weights in each column of the parameter matrix as the column score.

6 . The device according to claim 2 , wherein the processor is further configured to perform the zeroing by:
determining, for each parameter matrix, a threshold based on a determined columns dimension reduction amount and calculated column scores of the columns; and zeroing the parameters in the column, where the column score is less than the threshold, of each parameter matrix.

7 . The device according to claim 2 , wherein the processor is further configured to perform the zeroing by:
ranking the column scores of the columns of each parameter matrix based on magnitudes of the column scores; and zeroing, based on a determined column dimension reduction amount, the parameters in a predetermined number of columns, where the column scores are ranked one of high and low, of each parameter matrix.

8 . The device according to claim 2 , wherein the processor is further configured to retrain, according to parameter matrices with corresponding columns being zeroed, the neural network to obtain one or more column dimension reduced parameter matrices.

9 . The device according to claim 2 , wherein the processor is further configured to determine the columns dimension reduction amount of each of the one or more parameter matrixes, where a parameter matrix of parameter matrices, which is closer to an input layer, has a smaller column dimension reduction amount, and that a sum of the column dimension reduction amounts of all the parameter matrices meet the predetermined processing speed improvement target.

10 . The device according to claim 2 , wherein the processor is further configured to:
zero, according to the zeroed column of each parameter matrix, elements in a corresponding row of an input matrix corresponding to the parameter matrix; and retrain the neural network according to parameter matrices with corresponding columns being zeroed and at least one input matrix with corresponding rows being zeroed to obtain the at least one dimension reduced parameter matrix.

11 . The device according to claim 1 , wherein the processor is further configured to perform:
determining, according to another predetermined processing speed improvement target, a dimension reduction amount of each of the at least one dimension reduced parameter matrix obtained through retraining; re-preprocessing each parameter matrix based on the determined dimension reduction amount of the parameter matrix; and retraining, based on a result of the re-preprocessing, the neural network to obtain at least one parameter matrix with dimensions being reduced again to ensure the performance of the neural network meets the predetermined requirement, wherein the determining, the re-preprocessing and the retraining are performed repeatedly until at least one dimension reduced parameter matrix meeting a final processing speed improvement target is obtained.

12 . The device according to claim 1 , wherein the predetermined processing speed improvement target is determined where an effect on the performance of the neural network is within a tolerance range.

13 . The device according to claim 1 , wherein the neural network comprises a convolutional neural network (CNN).

14 . The device according to claim 1 , wherein in the case that the neural network is a convolutional neural network (CNN), the at least one parameter matrix represents parameter matrices of one or more convolution layers and/or a fully connection layer.

15 . A method for improving a processing speed of a neural network, the method comprising:
determining, according to a predetermined processing speed improvement target, a dimension reduction amount of each of at least one parameter matrix in the neural network obtained through training; preprocessing each parameter matrix based on the dimension reduction amount of the parameter matrix; and retraining the neural network based on a result of the preprocessing to obtain at least one dimension reduced parameter matrix to ensure performance of the neural network meets a predetermined requirement.

16 . The method according to claim 15 , wherein the dimension reduction amount represents a column dimension reduction amount of each parameter matrix, and the preprocessing further comprises:
calculating, for each parameter matrix, a column score of columns of the parameter matrix according to values of parameters in each column of the parameter matrix; and zeroing, for each parameter matrix, the parameters in a column of the parameter matrix the column score of which meets a predetermined condition, according to the column dimension reduction amount of the parameter matrix.

17 . The method according to claim 16 , wherein calculating the column score further comprises: calculating, for each parameter matrix, a sum of absolute values of the parameters in each column of the parameter matrix as the column score of the column.

18 . The method according to claim 16 , wherein calculating the column score further comprises: calculating, for each parameter matrix, the column score according to loss weights associated with the parameters in each column of the parameter matrix.

19 . The method according to claim 18 , wherein calculating the column score further comprises:
normalizing all of the parameters and the loss weights in each of the at least one parameter matrix; and calculating, for each parameter matrix, a sum of weighted sums of normalized parameters and normalized loss weights in each colum

Assignees

• Fujitsu Ltd

Inventors

• Wang Liuan
• Fan Wei
• Sun Jun

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title