Lossless exponent and lossy mantissa weight compression for training deep neural networks

Patent metadata

Field	Value
Publication number	US-11615301-B2
Application number	US-201916559241-A
Country	US
Kind code	B2
Filing date	Sep 3, 2019
Priority date	Sep 3, 2019
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

Systems, methods, and apparatuses are provided for compressing values. A plurality of parameters may be obtained from a memory, each parameter comprising a floating-point number that is used in a relationship between artificial neurons or nodes in a model. A mantissa value and an exponent value may be extracted from each floating-point number to generate a set of mantissa values and a set of exponent values. The set of mantissa values may be compressed to generate a mantissa lookup table (LUT) and a plurality of mantissa LUT index values. The set of exponent values may be encoded to generate an exponent LUT and a plurality of exponent LUT index values. The mantissa LUT, mantissa LUT index values, exponent LUT, and exponent LUT index values may be provided to one or more processing entities to train the model.

First claim

What is claimed is:

1. A system for compressing values, comprising:
one or more processors; and
one or more memory devices that store program code configured to be executed by the one or more processors, the program code comprising:
a floating-point number separator configured to:
obtain a plurality of parameters from a parameter memory, each parameter comprising a floating-point number that is used in a relationship between artificial neurons or nodes in a model;
extract a mantissa value and an exponent value from each floating-point number to generate a set of mantissa values and a set of exponent values;

a mantissa compressor configured to compress the set of mantissa values to generate a mantissa lookup table and a plurality of mantissa lookup table index values, each parameter being assigned one of the plurality of mantissa lookup table index values;
an exponent encoder configured to encode the set of exponent values to generate an exponent lookup table and a plurality of exponent lookup table index values, each parameter being assigned one of the plurality of exponent lookup table index values; and
a compressed parameter communicator configured to provide the mantissa lookup table, mantissa lookup table index values, exponent lookup table, and exponent lookup table values to at least one processing entity to train the model.

2. The system of claim 1 , wherein the at least one processing entity comprises at least one hardware accelerator, and
wherein the model comprises a deep-neural network.

3. The system of claim 2 , wherein the at least one processing entity is configured to:
generate a set of decompressed fixed-point values based at least on the mantissa lookup table, the mantissa lookup table index values, the exponent lookup table, and the exponent lookup table index values;
convert the set of decompressed fixed-point values into a set of decompressed floating-point parameters; and
train the deep neural network using the set of decompressed floating-point parameters.

4. The system of claim 1 , wherein the mantissa compressor is configured to compress the set of mantissa values to generate the mantissa lookup table by:
partitioning the set of mantissa values into a plurality of mantissa clusters, each cluster comprising a fixed-point cluster centroid; and
populating the mantissa lookup table with the fixed-point cluster centroids, each mantissa lookup table index value identifying a particular one of the fixed-point cluster centroids.

5. The system of claim 1 , wherein the encoded set of exponent values is lossless.

6. The system of claim 1 , wherein the mantissa compressor is configured to compress the set of mantissa values in parallel with the exponent encoder encoding the set of exponent values.

7. The system of claim 1 , wherein each floating-point number is one of a single-precision floating-point number or a double-precision floating-point number.

8. A method for compressing values, comprising:
obtaining a plurality of parameters from a parameter memory, each parameter comprising a floating-point number that is used in a relationship between artificial neurons or nodes in a model;
extracting a mantissa value and an exponent value from each floating-point number to generate a set of mantissa values and a set of exponent values;
compressing the set of mantissa values to generate a mantissa lookup table and a plurality of mantissa lookup table index values, each parameter being assigned one of the plurality of mantissa lookup table index values;
encoding the set of exponent values to generate an exponent lookup table and a plurality of exponent lookup table index values, each parameter being assigned one of the plurality of exponent lookup table index values; and
providing the mantissa lookup table, mantissa lookup table index values, exponent lookup table, and exponent lookup table values to at least one processing entity to train the model.

9. The method of claim 8 , wherein the at least one processing entity comprises at least one hardware accelerator, and
wherein the model comprises a deep-neural network.

10. The method of claim 9 , further comprising:
generating a set of decompressed fixed-point values based at least on the mantissa lookup table, the mantissa lookup table index values, the exponent lookup table, and the exponent lookup table index values;
converting the set of decompressed fixed-point values into a set of decompressed floating-point parameters; and
training the deep neural network using the set of decompressed floating-point parameters.

11. The method of claim 8 , wherein the compressing the set of mantissa values to generate the mantissa lookup table comprises:
partitioning the set of mantissa values into a plurality of mantissa clusters, each cluster comprising a fixed-point cluster centroid; and
populating the mantissa lookup table with the fixed-point cluster centroids, each mantissa lookup table index value identifying a particular one of the fixed-point cluster centroids.

12. The method of claim 8 , wherein the encoded set of exponent values is lossless.

13. The method of claim 8 , wherein the compressing the set of mantissa values is performed in parallel with the encoding the set of exponent values.

14. The method of claim 8 , wherein each floating-point number is one of a single-precision floating-point number or a double-precision floating-point number.

15. A device comprising:
a floating-point number separator circuit configured to:
obtain a plurality of parameters from a parameter memory, each parameter comprising a floating-point number that is used in a relationship between artificial neurons or nodes in a model; and
extract a mantissa value and an exponent value from each floating-point number to generate a set of mantissa values and a set of exponent values;

a mantissa compressor circuit configured to compress the set of mantissa values to generate a mantissa lookup table and a plurality of mantissa lookup table index values, each parameter being assigned one of the plurality of mantissa lookup table index values;
an exponent encoder circuit configured to encode the set of exponent values to generate an exponent lookup table and a plurality of exponent lookup table index values, each parameter being assigned one of the plurality of exponent lookup table index values; and
a compressed parameter outputting circuit configured to output the mantissa lookup table, mantissa lookup table index values, exponent lookup table, and exponent lookup table values for use by at least one processing entity to train the model.

16. The device of claim 15 , wherein the at least one processing entity comprises at least one hardware accelerator, and
wherein the model comprises a deep-neural network.

17. The device of claim 16 , wherein the at least one processing entity comprises circuitry configured to:
generate a set of decompressed fixed-point values based at least on the mantissa lookup table, the mantissa lookup table index values, the exponent lookup table, and the exponent lookup table index values;
convert the set of decompressed fixed-point values into a set of decompressed floating-point parameters; and
train the deep neural network using the set of decompressed floating-point parameters.

18. The device of claim 15 , wherein the mantissa compressor circuit is configured to:
partition the set of mantissa values into a plurality of mantissa clusters, each cluster comprising a fixed-point cluster centroid; and
populate the mantissa lookup table with the fixed-poin

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Xi Jinwen
• Pudipeddi Bharadwaj
• Tremblay Marc

Classifications

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06F7/483

  Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers {(G06F7/4806, G06F7/4824, G06F7/49, G06F7/491, G06F7/544 take precedence)} · CPC title

• H03M7/6023

  Parallelization · CPC title

• G06N3/063

  using electronic means · CPC title