Neural network quantization parameter determination method and related products

Patent metadata

Field	Value
Publication number	US-11675676-B2
Application number	US-201916622541-A
Country	US
Kind code	B2
Filing date	Sep 19, 2019
Priority date	Jun 12, 2019
Publication date	Jun 13, 2023
Grant date	Jun 13, 2023

Abstract

The present disclosure relates to a neural network quantization parameter determination method and related products. A board card in the related products includes a memory device, an interface device, a control device, and an artificial intelligence chip, in which the artificial intelligence chip is connected with the memory device, the control device, and the interface device respectively. The memory device is configured to store data, and the interface device is configured to transmit data between the artificial intelligence chip and an external device. The control device is configured to monitor the state of the artificial intelligence chip. The board card can be used to perform an artificial intelligence computation.

First claim

What is claimed is:

1. A method for quantizing data in a neural network using neural network quantization parameters, comprising:
obtaining an analyzing result of each type of the data to be quantized, wherein the data includes at least one type of neurons, weights, gradients, or biases of the neural network;
determining a corresponding quantization parameter according to the analyzing result of each type of the data to be quantized and a data bit width corresponding to the data to be quantized;
quantizing the data using the corresponding quantization parameter to obtain quantized data;
performing inverse quantization on the quantized data to obtain inverse quantized data, wherein a data format of the inverse quantized data is the same as that of the corresponding pre-quantized data; and
determining a quantization error based on the quantized data and the inverse quantized data.

2. The method of claim 1 , wherein the neural network operation process includes at least one operation of neural network training, neural network inference, or neural network fine-tuning,
wherein the analyzing result includes a maximum value and a minimum value, or includes a maximum absolute value, of each type of data to be quantized,
wherein the maximum absolute value is determined according to the maximum value and the minimum value of each type of data to be quantized, and
wherein the quantization parameter is determined according to the data bit width along with either the maximum value and the minimum value of each type of the data to be quantized or the maximum absolute value of each type of the data to be quantized.

3. The method of claim 1 , wherein the data bit width is adjusted according to the quantization error by comparing the quantization error with a threshold to obtain a comparison result, and adjusting the data bit width according to the comparison result, wherein the threshold includes at least one of a first threshold or a second threshold.

4. The method of claim 3 , wherein the adjusting of the data bit width includes:
increasing the data bit width if the quantization error is greater than or equal to the first threshold, or
reducing the data bit width if the quantization error is less than or equal to the second threshold, or
keeping the data bit width unchanged if the quantization error is between the first threshold and the second threshold.

5. The method of claim 3 , wherein a method for obtaining the quantization error includes:
determining a quantization interval according to the data bit width, and
determining the quantization error according to the quantization interval, the number of the quantized data, and the corresponding pre-quantized data.

6. The method of claim 3 , wherein the pre-quantized data is the data to be quantized in the weight update iteration within a target iteration interval, and wherein the target iteration interval includes at least one weight update iteration and the same quantization parameter is used in the quantization process within the same target iteration interval.

7. The method of claim 6 , wherein the determining the target iteration interval includes:
at a predicted time point, determining a variation trend value of a point position parameters of data to be quantized involved in the weight update iteration, wherein the predicted time point is used to determine whether the quantization parameter needs to be adjusted or not, and the predicted time point corresponds to the time point when the weight update iteration is completed, and
determining the corresponding target iteration interval according to the variation trend value of the point position parameter.

8. The method of claim 3 , wherein the pre-quantized data is data to be quantized involved in weight update iteration within a target iteration interval, wherein the target iteration interval includes at least one weight update iteration, and the same data bit width is used in the quantization process within the same target iteration interval.

9. The method of claim 8 , wherein the determining the target iteration interval includes:
at a predicted time point, determining a variation trend value of a point position parameter and a variation trend value of data bit width corresponding to the data to be quantized involved in the weight iteration process, wherein the predicted time point is configured to determine whether the data bit width needs to be adjusted or not, and the predicted time point corresponds to the time point when the weight update iteration is completed, and
determining the corresponding target iteration interval according to the variation trend value of the point position parameter and the variation trend value of the data bit width.

10. The method of claim 9 , wherein the variation trend value of the data bit width is determined according to the corresponding quantization error.

11. The method of claim 8 , wherein the determining of the target iteration interval includes:
at a predicted time point, determining a variation trend value of a point position parameter of data to be quantized involved in the weight update iteration, wherein the predicted time point is configured to determine whether the data bit width needs to be adjusted or not, and the predicted time point corresponds to the time point when the weight update iteration is completed, and
determining the corresponding target iteration interval according to the variation trend value of the point position parameter.

12. The method of claim 11 , wherein the predicted time point includes a first predicted time point, wherein the first predicted time point is determined according to the target iteration interval.

13. The method of claim 12 , wherein the predicted time point further includes a second predicted time point, wherein the second predicted time point is determined according to a curve of data variation range, wherein the curve of data variation range is obtained by analyzing the data variation range in the process of weight update iteration.

14. The method of claim 11 , wherein the variation trend value of the point position parameter is determined according to a moving average value of the point position parameter corresponding to a current predicted time point and a moving average value of the point position parameter corresponding to a previous predicted time point, or is determined according to the point position parameter corresponding to the current predicted time point and the moving average value of the corresponding point position parameter corresponding to the previous predicted time point.

15. The method of claim 14 , wherein the determining of a moving average value of a point position parameter corresponding to the current predicted time point includes:
determining the point position parameter corresponding to the current predicted time point according to a point position parameter corresponding to a previous predicted time point and an adjusted value of the data bit width,
adjusting a moving average value of a point position parameter corresponding to the previous predicted time point according to the adjusted value of the data bit width to obtain an adjusted result, and
determining the moving average value of the point position parameter corresponding to the current predicted time point according to the point position parameter corresponding to the current predicted time point and the adjusted result.

16. The method of claim 14 , wherein the determining of the moving average value of the point position parameter corresponding to the current predicted time point include:
determining an int

Assignees

• Shanghai Cambricon Inf Tech Co Ltd

Inventors

• Liu Shaoli
• Meng Xiaofu
• Zhang Xishan
• Guo Jiaming

Classifications

• G06F2201/865

  Monitoring of software · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06F11/1476 Primary

  in neural networks · CPC title

• G06F2201/81

  Threshold · CPC title

• G06N3/045

  Combinations of networks · CPC title