Data processing method and apparatus

Patent metadata

Field	Value
Publication number	US-12519486-B2
Application number	US-202418412995-A
Country	US
Kind code	B2
Filing date	Jan 15, 2024
Priority date	Aug 25, 2021
Publication date	Jan 6, 2026
Grant date	Jan 6, 2026

Abstract

This application provides a data processing method and apparatus. The method is applied to a storage system. The storage system includes a storage apparatus and a processing apparatus. The method is performed by the processing apparatus. The method includes: obtaining a tiered storage feature and a data feature of first data, where the tiered storage feature includes at least one of the following features: an importance, an access frequency, and a retention time, and the data feature includes at least one of the following features: a data type, a data dimension, a data size, or a data content feature; determining a first compression algorithm based on the tiered storage feature and the data feature; and compressing the first data based on the first compression algorithm, to obtain compressed data.

First claim

What is claimed is:

1 . A data processing method performed by a processing apparatus of a storage system, the method comprising:
obtaining a tiered storage feature and a data feature of first data, wherein the tiered storage feature comprises at least one of the following features: an importance, an access frequency, or a retention time, and the data feature comprises at least one of the following features: a data type, a data dimension, a data size, or a data content feature; determining a first compression algorithm based on the tiered storage feature and the data feature; and compressing the first data based on the first compression algorithm, to obtain compressed data, wherein the first data is data that has been compressed, and the compressing the first data comprises: decompressing the first data to obtain intermediate data; saving a first operation parameter used during the decompression; and compressing the intermediate data based on the first compression algorithm, to obtain the compressed data.

2 . The method of claim 1 , wherein the determining the first compression algorithm based on the tiered storage feature and the data feature comprises:
determining a compression algorithm performance parameter based on the tiered storage feature; and determining the first compression algorithm based on the compression algorithm performance parameter and the data feature.

3 . The method of claim 2 , wherein before the determining the first compression algorithm, the method further comprises:
determining, based on the tiered storage feature, that the first data needs to be compressed.

4 . The method of claim 2 , wherein the compression algorithm performance parameter comprises at least one of the following: a compression ratio or a throughput.

5 . The method of claim 1 , the method further comprising:
decompressing the compressed data to obtain the intermediate data; and compressing the intermediate data based on the first operation parameter, to obtain the first data.

6 . The method of claim 1 , the method further comprising:
decompressing the compressed data to obtain the first data.

7 . The method of claim 1 , the method further comprising:
storing the compressed data in the storage apparatus.

8 . A non-transitory computer-readable storage medium, storing instructions, wherein when the instructions are run on a computer, the computer is enabled to perform a data processing method comprising:
obtaining a tiered storage feature and a data feature of first data, wherein the tiered storage feature comprises at least one of the following features: an importance, an access frequency, or a retention time, and the data feature comprises at least one of the following features: a data type, a data dimension, a data size, or a data content feature; determining a first compression algorithm based on the tiered storage feature and the data feature; and compressing the first data based on the first compression algorithm, to obtain compressed data, wherein the first data is data that has been compressed, and the compressing the first data comprises: decompressing the first data to obtain intermediate data; saving a first operation parameter used during the decompression; and compressing the intermediate data based on the first compression algorithm, to obtain the compressed data.

9 . The non-transitory computer-readable storage medium of claim 8 , wherein the determining the first compression algorithm based on the tiered storage feature and the data feature comprises:
determining a compression algorithm performance parameter based on the tiered storage feature; and determining the first compression algorithm based on the compression algorithm performance parameter and the data feature.

10 . The non-transitory computer-readable storage medium of claim 9 , wherein before the determining the first compression algorithm, the method further comprises:
determining, based on the tiered storage feature, that the first data needs to be compressed.

11 . The non-transitory computer-readable storage medium of claim 8 , wherein the method further comprises:
decompressing the compressed data to obtain the intermediate data; and compressing the intermediate data based on the first operation parameter, to obtain the first data.

12 . The non-transitory computer-readable storage medium of claim 8 , wherein the method further comprises:
decompressing the compressed data to obtain the first data.

13 . A computer device, wherein the computer device comprises a processor and a memory;
the memory is configured to store computer program instructions; and the processor executes the computer program instructions in the memory to perform a method comprising: obtaining a tiered storage feature and a data feature of first data, wherein the tiered storage feature comprises at least one of the following features: an importance, an access frequency, or a retention time, and the data feature comprises at least one of the following features: a data type, a data dimension, a data size, or a data content feature; determining a first compression algorithm based on the tiered storage feature and the data feature; and compressing the first data based on the first compression algorithm, to obtain compressed data, wherein the first data is data that has been compressed, and the compressing the first data comprises: decompressing the first data to obtain intermediate data; saving a first operation parameter used during the decompression; and compressing the intermediate data based on the first compression algorithm, to obtain the compressed data.

14 . The computer device of claim 13 , wherein the determining the first compression algorithm based on the tiered storage feature and the data feature comprises:
determining a compression algorithm performance parameter based on the tiered storage feature; and determining the first compression algorithm based on the compression algorithm performance parameter and the data feature.

15 . The computer device of claim 14 , wherein before the determining the first compression algorithm, the method further comprises:
determining, based on the tiered storage feature, that the first data needs to be compressed.

16 . The computer device of claim 14 , wherein the method further comprises:
decompressing the compressed data to obtain the intermediate data; and compressing the intermediate data based on the first operation parameter, to obtain the first data.

17 . The computer device of claim 13 , wherein the method further comprises:
decompressing the compressed data to obtain the first data.

Assignees

• Huawei Tech Co Ltd

Inventors

• Wang Zhibing
• Liu Zhenbao

Classifications

• G06F3/0661

  Format or protocol conversion arrangements · CPC title

• G06F3/0643

  Management of files · CPC title

• H03M7/6029

  Pipelining · CPC title

• G06F3/0649

  Lifecycle management · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title