Apparatus and Methods for Neural Network Operations Supporting Floating Point Numbers of Short Bit Length

Patent metadata

Field	Value
Publication number	US-2019079727-A1
Application number	US-201816174084-A
Country	US
Kind code	A1
Filing date	Oct 29, 2018
Priority date	Apr 28, 2016
Publication date	Mar 14, 2019
Grant date	—

Abstract

Aspects for neural network operations with floating-point number of short bit length are described herein. The aspects may include a neural network processor configured to process one or more floating-point numbers to generate one or more process results. Further, the aspects may include a floating-point number converter configured to convert the one or more process results in accordance with at least one format of shortened floating-point numbers. The floating-point number converter may include a pruning processor configured to adjust a length of a mantissa field of the process results and an exponent modifier configured to adjust a length of an exponent field of the process results in accordance with the at least one format.

First claim

We claim:

1 . An apparatus for neural network operations, comprising:
a neural network processor configured to process one or more floating-point numbers to generate one or more process results; and a floating-point number converter configured to convert the one or more process results in accordance with at least one format of shortened floating-point numbers, wherein the floating-point number converter includes
a pruning processor configured to adjust a length of a mantissa field of the process results, and
an exponent modifier configured to adjust a length of an exponent field of the process results in accordance with the at least one format.

2 . The apparatus of claim 1 , further comprising a floating-point number analyzing processor configured to determine the at least one format of the shortened floating-point numbers, wherein the floating-point number analyzing processor includes:
a data extractor configured to collect one or more categories of the floating-point numbers; a data analyzer configured to statistically analyze the one or more categories of the floating-point numbers to determine a data range for each of the one or more categories of the floating-point numbers and a distribution pattern for each of the one or more categories over one or more subranges of the data range; and a format determiner configured to determine the at least one format of shortened floating-point numbers for the one or more categories.

3 . The apparatus of claim 2 , wherein the format determiner is configured to determine one of the at least one format of shortened floating-point numbers for each of the one or more categories.

4 . The apparatus of claim 1 , wherein each of the at least one format further includes a length limit of an exponent field stored in one or more register files.

5 . The apparatus of claim 1 , wherein the at least one format at least includes an offset value and a bias value, and wherein the offset value and the bias value are stored in one or more register files.

6 . The apparatus of claim 1 , further comprising a data cache configured to store the one or more process results.

7 . The apparatus of claim 1 , wherein the pruning processor includes a random trimmer configured to:
adjust the length of the mantissa field of the process results to a first length in accordance with a first probability; and adjust the length of the mantissa field of the process results to a second length in accordance with a second probability.

8 . The apparatus of claim 1 , wherein the pruning processor includes a half-adjust trimmer configured to:
add an integer value to the mantissa field if a fraction indicated by the mantissa field is not less than one-half of a smallest positive integer representable by the format of the shortened floating-point numbers, wherein the integer value is equal to the smallest positive integer representable by the format of the shortened floating-point numbers; and clear the fraction indicated by the mantissa field if the fraction is less than one-half of the smallest positive integer representable by the format of the shortened floating-point numbers.

9 . The apparatus of claim 1 , wherein the pruning processor includes a round-up trimmer configured to round up the mantissa field to a smallest positive integer that is greater than the process result.

10 . The apparatus of claim 1 , wherein the pruning processor includes a round-down trimmer configured to round down the mantissa field to a greatest positive integer that is less than the process result.

11 . The apparatus of claim 1 , wherein the pruning processor includes a cut-off trimmer configured to discard mantissa digits that exceed a length of a mantissa field in accordance with the at least one format of shortened floating-point numbers.

12 . A method for neural network operations, comprising:
processing, by a neural network processor, one or more floating-point numbers to generate one or more process results; converting, by a floating-point number converter, the one or more process results in accordance with at least one format of shortened floating-point numbers, wherein the converting includes:
adjusting, by a pruning processor, a length of a mantissa field of the process results, and
adjusting, by an exponent modifier a length of an exponent filed of the process results in accordance with the at least one format.

13 . The method of claim 12 , further comprising:
collecting, by a data extractor, one or more categories of floating-point numbers; statistically analyzing, by a data analyzer, the one or more categories of the floating-point numbers to determine a data range for each of the one or more categories of floating-point numbers and a distribution pattern for each of the one or more categories over one or more subranges of the data range; and determining, by a format determiner, the at least one format of shortened floating-point numbers for the one or more categories.

14 . The method of claim 13 , further comprising determining, by the format determiner, one of the at least one format for shortened floating-point numbers for each of the one or more categories.

15 . The method of claim 13 , wherein each of the at least one format further includes a length limit of an exponent field stored in one or more register files.

16 . The method of claim 13 , wherein the at least one format at least includes an offset value and a bias value, and wherein the offset value and the bias value are stored in one or more register files.

17 . The method of claim 12 , wherein the adjusting the length of the mantissa field further comprises:
adjusting, by a random trimmer of the pruning processor, the length of the mantissa field of the process results to a first length in accordance with a first probability; and adjusting, by the random trimmer of the pruning processor, the length of the mantissa field of the process results to a second length in accordance with a second probability.

18 . The method of claim 12 , wherein the adjusting the length of the mantissa field further comprises:
adding, by a half-adjust trimmer of the pruning processor, one to a second least significant bit of the mantissa field if a least significant digit is not less than one-half of a smallest positive integer representable by the format of the shortened floating-point numbers; and clearing, by the half-adjust trimmer of the pruning processor, the least significant digit if the least significant digit is less than one-half of a smallest positive integer representable by the format of the shortened floating-point numbers.

19 . The method of claim 12 , wherein the adjusting the length of the mantissa field further comprises rounding up, by a round-up trimmer of the pruning processor, the mantissa field to a smallest positive integer that is greater than the process result.

20 . The method of claim 12 , wherein the adjusting the length of the mantissa field further comprises rounding down, by a round-down trimmer, the mantissa field to a greatest positive integer that is less than the process result.

21 . The method of claim 12 , wherein the adjusting the length of the mantissa field further comprises discarding, by a cut-off trimmer, exponent digits that exceed a length of the mantissa field in accordance with the at least one format of shortened floating-point numbers.

Assignees

• Cambricon Tech Corp Ltd

Inventors

• Chen Tianshi
• Liu Shaoli
• Guo Qi
• Chen Yunji

Classifications

• G06F2207/4824

  Neural networks · CPC title

• G06F9/44

  Arrangements for executing specific programs · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/02

  Neural networks · CPC title

• G06F7/575 Primary

  Basic arithmetic logic units, i.e. devices selectable to perform either addition, subtraction or one of several logical operations, using, at least partially, the same circuitry · CPC title