Lossless machine learning activation value compression

Patent metadata

Field	Value
Publication number	US-11394396-B2
Application number	US-202017032962-A
Country	US
Kind code	B2
Filing date	Sep 25, 2020
Priority date	Sep 25, 2020
Publication date	Jul 19, 2022
Grant date	Jul 19, 2022

Abstract

Techniques are disclosed for compressing data. The techniques include identifying, in data to be compressed, a first set of values, wherein the first set of values include a first number of two or more consecutive identical non-zero values; including, in compressed data, a first control value indicating the first number of non-zero values and a first data item corresponding to the consecutive identical non-zero values; identifying, in the data to be compressed, a second value having an exponent value included in a defined set of exponent values; including, in the compressed data, a second control value indicating the exponent value and a second data item corresponding to a portion of the second value other than the exponent value; and including, in the compressed data, a third control value indicating a third set of one or more consecutive zero values in the data to be compressed.

First claim

What is claimed is:

1. A method for compressing data, the method comprising:
identifying, in data to be compressed, a first set of values, wherein the first set of values include a first number of two or more consecutive identical non-zero values;
including, in compressed data, a first control value indicating the first number of non-zero values and a first data item corresponding to the consecutive identical non-zero values;
identifying, in the data to be compressed, a second value having an exponent value included in a defined set of exponent values;
including, in the compressed data, a second control value indicating the exponent value and a second data item corresponding to a portion of the second value other than the exponent value; and
including, in the compressed data, a third control value indicating a third set of one or more consecutive zero values in the data to be compressed.

2. The method of claim 1 , further comprising:
including, in the compressed data, a fourth control value indicating a value for which compression does not occur; and
including, in the compressed data, the value, as a third data item.

3. The method of claim 1 , further comprising:
decompressing the compressed data by expanding data items of the compressed data according to control values of the compressed data.

4. The method of claim 3 , wherein expanding the data items includes duplicating one or more data items stored as repeating items, and adding a full exponent value for data items stored as exponent-compressed.

5. The method of claim 1 , further comprising:
including, within a consecutive control values portion of a compressed block, the first control value, the second control value, and the second value.

6. The method of claim 1 , further comprising:
including, within a consecutive data items portion of a compressed block, the first data item and the second data item.

7. The method of claim 1 , further comprising:
adding data for one or more values for compression until an input uncompressed block is completely processed.

8. The method of claim 1 , further comprising:
adding, into a compressed block, a data offset indicating a position of a data items portion as an offset from a beginning of the compressed block.

9. The method of claim 1 , further comprising:
modifying at least one value of the defined set of exponent values.

10. A memory device, comprising:
a memory; and
a memory interface, configured to:
identify, in data to be compressed, a first set of values, wherein the first set of values include a first number of two or more consecutive identical non-zero values;
include, in compressed data, a first control value indicating the first number of non-zero values and a first data item corresponding to the consecutive identical non-zero values;
identify, in the data to be compressed, a second value having an exponent value included in a defined set of exponent values;
include, in the compressed data, a second control value indicating the exponent value and a second data item corresponding to a portion of the second value other than the exponent value; and
include, in the compressed data, a third control value indicating a third set of one or more consecutive zero values in the data to be compressed.

11. The memory device of claim 10 , wherein the memory interface is further configured to:
include, in the compressed data, a fourth control value indicating a value for which compression does not occur; and
include, in the compressed data, the value, as a third data item.

12. The memory device of claim 10 , wherein the memory interface is further configured to:
decompress the compressed data by expanding data items of the compressed data according to control values of the compressed data.

13. The memory device of claim 12 , wherein expanding the data items includes duplicating one or more data items stored as repeating items, and adding a full exponent value for data items stored as exponent-compressed.

14. The memory device of claim 10 , wherein the memory interface is further configured to:
including, within a consecutive control values portion of a compressed block, the first control value, the second control value, and the second value.

15. The memory device of claim 10 , wherein the memory interface is further configured to:
include, within a consecutive data items portion of a compressed block, the first data item and the second data item.

16. The memory device of claim 10 , wherein the memory interface is further configured to:
add data for one or more values for compression until an input uncompressed block is completely processed.

17. The memory device of claim 10 , wherein the memory interface is further configured to:
adding, into a compressed block, a data offset indicating a position of a data items portion as an offset from a beginning of the compressed block.

18. The memory device of claim 10 , wherein the memory interface is further configured to:
modify at least one value of the defined set of exponent values.

19. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:
identify, in data to be compressed, a first set of values, wherein the first set of values include a first number of two or more consecutive identical non-zero values;
include, in compressed data, a first control value indicating the first number of non-zero values and a first data item corresponding to the consecutive identical non-zero values;
identify, in the data to be compressed, a second value having an exponent value included in a defined set of exponent values;
include, in the compressed data, a second control value indicating the exponent value and a second data item corresponding to a portion of the second value other than the exponent value; and
include, in the compressed data, a third control value indicating a third set of one or more consecutive zero values in the data to be compressed.

20. The non-transitory computer-readable medium of claim 19 , wherein the memory interface is further configured to:
include, in the compressed data, a fourth control value indicating a value for which compression does not occur; and
include, in the compressed data, the value, as a third data item.

Assignees

• Advanced Micro Devices Inc

Inventors

• Argade Pramod Vasant
• Sakharshete Swapnil P
• Peroni Daniel N

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/063

  using electronic means · CPC title

• H03M7/3091

  Data deduplication · CPC title

• H03M7/24 Primary

  Conversion to or from floating-point codes · CPC title