Compression of neural network activation data

Patent metadata

Field	Value
Publication number	US-11948069-B2
Application number	US-201916518444-A
Country	US
Kind code	B2
Filing date	Jul 22, 2019
Priority date	Jul 22, 2019
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

A processor arranged to compress neural network activation data comprising an input module for obtaining neural network activation data. The processor also comprises a block creation module arranged to split the neural network activation data into a plurality of blocks; and a metadata generation module for generating metadata associated with at least one of the plurality of blocks. Based on the metadata generated a selection module selects a compression scheme for each of the plurality of blocks, and a compression module for applying the selected compression scheme to the corresponding block to produce compressed neural network activation data. An output module is also provided for outputting the compressed neural network activation data.

First claim

What is claimed is:

1. A processor arranged to compress neural network activation data comprising:
an input module for obtaining uncompressed neural network activation data from memory;
a block creation module arranged to split the uncompressed neural network activation data into a plurality of blocks of uncompressed neural network activation data;
a metadata generation module, comprising hardware logic, for generating metadata associated with each of the plurality of blocks of uncompressed neural network activation data, the metadata comprising statistics calculated by:
a zero-counter of the metadata generation module, arranged to calculate a number of zero point values of each of the plurality of blocks of uncompressed neural network activation data;
a unique non-zero counter of the metadata generation module, arranged to calculate a number of unique non-zero point values of each of the plurality of blocks of uncompressed neural network activation data; and
a non-zero counter of the metadata generation module, arranged to calculate a number of non-zero point values of each of the plurality of blocks of uncompressed neural network activation data;

a selection module for selecting a compression scheme for each of the plurality of blocks of uncompressed neural network activation data, wherein the compression scheme is selected based on at least the statistics associated with each block of the uncompressed neural network activation data;
a compression module for applying the selected compression scheme to the corresponding block of uncompressed neural network activation data to produce compressed neural network activation data; and
an output module for outputting the compressed neural network activation data.

2. The processor arranged to compress neural network activation data according to claim 1 , further comprising a combination module for combining a plurality of outputs of the compression module.

3. The processor arranged to compress neural network activation data according to claim 1 , wherein the processor is a neural processing unit.

4. The processor arranged to compress neural network activation data of claim 1 , wherein the processor is any of:
an image processor;
a central processing unit; and
a graphics processing unit.

5. The processor arranged to compress neural network activation data according to claim 4 , wherein the output module is arranged to output the compressed neural network activation data to at least one neural processing unit.

6. The processor arranged to compress neural network activation data according to claim 1 , wherein the compression module comprises at least one of:
a masking compression unit for applying a masking compression technique;
a look-up table compression unit for applying a look-up table compression technique;
a value packing compression unit for applying a value packing compression technique; and
a position packing compression unit for applying a position packing compression technique.

7. The processor arranged to compress neural network activation data according to claim 1 , further comprising a plurality of metadata generation modules, selection modules and compression modules, and the wherein output module is arranged to combine the compressed neural network activation data of each of the compression modules into a single compressed output.

8. The processor arranged to compress neural network activation data according to claim 1 , wherein the output module comprises a memory arranged to store the compressed output associated with each block, and wherein the output module is arranged to combine the compressed neural network activation data associated with each of the plurality of blocks.

9. A method for compressing neural network activation data, the method comprising the steps of:
obtaining uncompressed neural network activation data from memory;
splitting the uncompressed neural network activation data into a plurality of blocks;
generating metadata for each of the plurality of blocks, the metadata comprising statistics indicating:
a number of zero point values of the block of neural network activation data;
a number of unique non-zero point values of the block of neural network activation data; and
a number of non-zero point values of the block of neural network activation data;

selecting a compression scheme for each of the plurality of blocks of uncompressed neural network activation data, wherein the compression scheme is selected based on at least the statistics associated with each block of the uncompressed neural network activation data;
applying the selected compression schemes to the corresponding block, to produce compressed neural network activation data; and
outputting the compressed neural network activation data.

10. The method for compressing neural network activation data according to claim 9 , wherein each block is an 8×8 block of data.

11. The method for compressing neural network activation data according to claim 9 , wherein the compressions schemes are any of:
a masking compression technique;
a look-up table compression technique;
a value packing compression technique; and
a position packing compression technique.

12. A system for compressing neural network activation data, the system comprising a processor arranged to compress neural network activation data according to claim 1 .

13. The system for compressing neural network activation data according to claim 12 , wherein the processor is a neural processing unit.

14. The system for compressing neural network activation data according to claim 12 , wherein the processor is any of an image processor; a central processing unit; and a graphics processing unit, and wherein the system further comprises a neural processing unit for receiving compressed neural network activation data output by the processor.

Assignees

• Advanced Risc Mach Ltd

Inventors

• Meng Lingchuan
• Brothers Iii John Wakefield
• Olson Jens
• Smolens Jared Corey
• Kunze Eric
• Bratt Ian Rudolf

Classifications

• G06N3/063 Primary

  using electronic means · CPC title

• H03M7/70

  Type of the data to be coded, other than image and sound · CPC title

• H03M7/3066 Primary

  by means of a mask or a bit-map · CPC title

• H03M7/6047

  Power optimization with respect to the encoder, decoder, storage or transmission · CPC title

• H03M7/6064

  Selection of Compressor · CPC title