Operation accelerator and compression method

Patent metadata

Field	Value
Publication number	US-12367165-B2
Application number	US-202418601409-A
Country	US
Kind code	B2
Filing date	Mar 11, 2024
Priority date	Sep 30, 2018
Publication date	Jul 22, 2025
Grant date	Jul 22, 2025

Abstract

The present disclosure provides example operation accelerators and compression methods. One example operation accelerator includes a storage configured to store first input data, weight data, and a control instruction, and an operation circuit connected to the storage and configured to perform matrix multiplication on the first input data and the weight data, to obtain a computation result. The operation accelerator further includes a compression module configured to compress the computation result to obtain compressed data, as well as a controller connected to the storage and configured to obtain the control instruction from the storage, and when the control instruction includes instructions to compress the computation result, control the compression module to compress the computation result to obtain the compressed data. The operation accelerator further includes a direct memory access controller connected to the compression module and configured to store the compressed data in a memory outside the operation accelerator.

First claim

What is claimed is:

1. An operation accelerator, comprising:
a storage, configured to store first input data, weight data, and a control instruction;
an operation circuit connected to the storage, configured to perform matrix multiplication on the first input data and the weight data, to obtain a computation result, wherein the control instruction instructs whether to compress the computation result;
a compression module, configured to compress the computation result to obtain compressed data;
a controller connected to the storage and configured to:
obtain the control instruction from the storage; and
when the control instruction comprises instructions to compress the computation result, control the compression module to compress the computation result to obtain the compressed data; and

a direct memory access controller (DMAC) connected to the compression module, configured to store the compressed data in a memory outside the operation accelerator.

2. The operation accelerator according to claim 1 , further comprising:
a decompression module connected to the DMAC and the storage, configured to:
receive the compressed data obtained by the DMAC from the memory;
decompress the compressed data; and
store the decompressed data in the storage as second input data; and

wherein the operation circuit is further configured to obtain the second input data from the storage to perform matrix multiplication.

3. The operation accelerator according to claim 2 , wherein the control instruction is used to instruct whether to decompress the computation result; and
the controller is configured to:
parse the control instruction; and
when the control instruction comprises instructions to decompress the computation result, control the decompression module to decompress the obtained compressed data.

4. The operation accelerator according to claim 3 , wherein:
the storage is further configured to store the computation result of the operation circuit; and
the controller is further configured to, when the control instruction skips instructing to compress or decompress the computation result:
control the DMAC to store the computation result in the storage into the memory; and
control the DMAC to store the computation result in the memory into the storage.

5. The operation accelerator according to claim 2 , wherein:
the control instruction is used to instruct whether to decompress the computation result;
the controller is configured to distribute the control instruction to the compression module and the decompression module;
the compression module is configured to:
parse the control instruction; and
when the control instruction comprises instructions to compress the computation result, compress the computation result to obtain the compressed data; and

the decompression module is configured to:
parse the control instruction; and
when the control instruction comprises instructions to decompress the computation result, decompress the obtained compressed data.

6. The operation accelerator according to claim 5 , wherein:
the compression module is further configured to, when the control instruction skips instructing to compress the computation result, control the DMAC to store the computation result in the memory; and
the decompression module is further configured to, when the control instruction skips instructing to decompress the computation result, control the DMAC to store the computation result in the memory into the storage.

7. The operation accelerator according to claim 2 , wherein the compression module comprises a fragmentation module and at least one compression engine, wherein:
the fragmentation module is configured to perform fragmentation processing on the computation result to obtain at least one sub-computation result; and
each compression engine in the at least one compression engine is configured to compress a respective sub-computation result in the at least one sub-computation result to obtain respective sub-compressed data, wherein a sum of all the respective sub-compressed data generated by each compression engine in the at least one compression engine forms the compressed data.

8. The operation accelerator according to claim 7 , wherein each compression engine in the at least one compression engine is configured to:
compress the respective sub-computation result to obtain a respective sub-compression result;
compare size of the respective sub-compression result with size of the respective sub-computation result; and
when the size of the respective sub-compression result is greater than the size of the respective sub-computation result, use the respective sub-computation result as the respective sub-compressed data; or
when the size of the respective sub-compression result is not greater than the size of the respective sub-computation result, use the respective sub-compression result as the respective sub-compressed data.

9. The operation accelerator according to claim 8 , wherein each compression engine in the at least one compression engine is further configured to:
when the size of the respective sub-compression result is greater than the size of the respective sub-computation result:
generate a respective compression failure identifier corresponding to the respective sub-compressed data; and
store the respective compression failure identifier in the memory; and

when the size of the respective sub-compression result is not greater than the size of the respective sub-computation result:
generate a respective compression success identifier corresponding to the respective sub-compressed data; and
store the respective compression success identifier in the memory.

10. The operation accelerator according to claim 9 , wherein the decompression module is configured to:
receive the respective sub-compressed data from the memory; and
when an identifier corresponding to the respective sub-compressed data is the respective compression failure identifier, store the respective sub-compressed data into the storage as the second input data; or
when the identifier corresponding to the respective sub-compressed data is the respective compression success identifier:
decompress, as respective sub-decompressed data, the respective sub-compressed data; and
store the respective sub-decompressed data into the storage as the second input data.

11. A compression method, wherein the compression method is applied to an operation accelerator, the operation accelerator comprises a storage, and the method comprises:
obtaining a computation result by performing matrix multiplication on first input data and weight data obtained from the storage;
obtaining a control instruction, wherein the control instruction instructs whether to compress the computation result; and
when the control instruction comprises instructions to compress the computation result:
obtaining compressed data by compressing the computation result; and
storing the compressed data in a memory outside the operation accelerator.

12. The method according to claim 11 , further comprising:
obtaining the compressed data from the memory;
decompressing the compressed data;
storing the decompressed data into the storage as second input data; and
performing matrix multiplication on the second input data obtained from the storage.

13. The compression method according to claim 12 , wherein the control instruction is used to instruct whether to decompress the computation result, and the method further comprises:
parsing the control instruction; and
wherein the decompressing the compressed data comprises:
when the control instruction comprises

Assignees

• Huawei Tech Co Ltd

Inventors

• Liu Baoqing
• Liu Hu
• Chen Qinglong

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• H03M7/30

  Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/048

  Activation functions · CPC title