Data transmission method and apparatus

Patent metadata

Field	Value
Publication number	US-10979520-B2
Application number	US-201815979951-A
Country	US
Kind code	B2
Filing date	May 15, 2018
Priority date	Nov 16, 2015
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

Embodiments of the present invention relate to the database field, and in particular, to a data transmission method and apparatus, so as to reduce overheads for data transmission between nodes in a distributed database and lighten network load. In the embodiments of the present invention, a DN determines a column that has a distribution rule of parameters in the column; the DN determines, according to a data type of the parameters of the column and the distribution rule of the parameters in the column, a compression algorithm corresponding to the column; the DN compresses the column using the compression algorithm; and the DN sends a compressed column to a target node. In this way, the overheads for the data transmission between the nodes are reduced, and the network load is lightened.

First claim

What is claimed is:

1. A data transmission method, comprising:
finding, by a data node (DN) according to a received query command, data that matches one or more keywords comprised in the query command; and
determining, by a data node (DN), a column corresponding to the keywords from the data, wherein the column comprises multiple parameters, values of at least two parameters in the multiple parameters are equal;
determining, by the DN according to a data type of the parameters of the column and a distribution rule of the parameters in the column, a compression algorithm corresponding to the column;
compressing, by the DN, data of the column using the compression algorithm; and
sending, by the DN, a compressed column to a target node;
wherein the determined compression algorithm corresponding to the column is a run-length encoding (RLE) compression algorithm when the data type of the parameters of the column is a data type that supports the RLE compression algorithm; and the determined compression algorithm corresponding to the column is a dictionary compression algorithm when the data type of the parameters of the column is a data type that supports the dictionary compression algorithm.

2. The method according to claim 1 , further comprising:
sending, by the DN, indication information to the target node; wherein
the indication information is used to indicate the compression algorithm corresponding to the column, and the indication information is used by the target node to decompress the compressed data to obtain the data.

3. The method according to claim 1 , further comprising:
sorting, by the DN according to values of the parameters of the column and on a row-by-row basis, N-row data corresponding to the column, wherein the parameters in the column are in a one-to-one correspondence with rows of the N-row data; wherein
when the data type of the parameters of the column is a data type that supports the RLE compression algorithm and the distribution rule of the column is sorted according to the values of the parameters of the column, the compression algorithm corresponding to the column is the RLE compression algorithm.

4. The method according to claim 1 , further comprising:
determining, by the DN according to values of the parameters of the column, parameters whose values are equal from the parameters of the column; and
arranging together, by the DN, multiple-row data corresponding to the determined multiple parameters whose values are equal, wherein the parameters in the column are in a one-to-one correspondence with rows of N-row data; wherein
when the data type of the parameters of the column is a data type that supports the RLE compression algorithm and the distribution rule of the column is arranged together the multiple-row data corresponding to the determined multiple parameters whose values are equal, the compression algorithm corresponding to the column is the RLE compression algorithm.

5. A data transmission apparatus, comprising:
a processor, a transceiver, and a memory coupled to the processor, wherein the memory is configured to store an computer-readable program, and the processor is configured to execute the program, so as to:
find, according to a received query command, data that matches one or more keywords comprised in the query command; and
determine a column corresponding to the keywords from the data, wherein the column comprises multiple parameters, values of at least two parameters in the multiple parameters are equal;
determine, according to a data type of the parameters of the column and a distribution rule of the parameters in the column, a compression algorithm corresponding to the column, and
compress data of the column using the compression algorithm; and

the transceiver is configured to send, under control of the processor, the compressed column to a target node;
wherein the determined compression algorithm corresponding to the column is a run-length encoding (RLE) compression algorithm when the data type of the parameters of the column is a data type that supports the RLE compression algorithm; and the determined compression algorithm corresponding to the column is a dictionary compression algorithm when the data type of the parameters of the column is a data type that supports the dictionary compression algorithm.

6. The apparatus according to claim 5 wherein the transceiver is further configured to:
send indication information to the target node, wherein
the indication information is used to indicate the compression algorithm corresponding to the column, and the indication information is used by the target node to decompress the compressed data to obtain the data.

7. The apparatus according to claim 5 , wherein the processor is further configured to:
sort according to values of the parameters of the column and on a row-by-row basis, N-row data corresponding to the column, wherein the parameters in the column are in a one-to-one correspondence with rows of the N-row data; and
wherein when the data type of the parameters of the column is a data type that supports the RLE compression algorithm and the distribution rule of the column is sorted according to the values of the parameters of the column, the compression algorithm corresponding to the column is the RLE compression algorithm.

8. The apparatus according to claim 5 , wherein the processor is further configured to:
determine according to values of the parameters of the column, parameters whose values are equal from the parameters of the column;
arrange together multiple-row data corresponding to the determined multiple parameters whose values are equal, wherein the parameters in the column are in a one-to-one correspondence with rows of N-row data; and
wherein when the data type of the parameters of the column is a data type that supports the RLE compression algorithm and the distribution rule of the column is arranged together the multiple-row data corresponding to the determined multiple parameters whose values are equal, the compression algorithm corresponding to the column is the RLE compression algorithm.

9. A non-transitory computer-readable medium storing computer instructions for data transmission, which when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:
finding, according to a received query command, data that matches one or more keywords comprised in the query command; and
determining a column corresponding to the keywords from the data;
determining, according to a data type of the parameters of the column and a distribution rule of the parameters in the column, a compression algorithm corresponding to the column;
compressing data of the column using the compression algorithm; and
sending a compressed data of the column to a target node;
wherein the determined compression algorithm corresponding to the column is a run length encoding (RLE) compression algorithm when the data type of the parameters of the column is a data type that supports the RLE compression algorithm; and the determined compression algorithm corresponding to the column is a dictionary compression algorithm when the data type of the parameters of the column is a data type that supports the dictionary compression algorithm.

10. The computer-readable medium according to claim 9 , wherein the method further comprises:
sending, by the DN, indication information to the target node, wherein
the indication information is used to indicate the compression algorithm corresponding to the column, and the indication information is used by the target node to decompress the compressed data to obtain the data.

11. The computer-readable m

Assignees

• Huawei Tech Co Ltd

Inventors

• Huang Lin
• Wang Chuanting

Classifications

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• H03M7/30 Primary

  Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction · CPC title

• H04L67/5651 Primary

  Reducing the amount or size of exchanged application data · CPC title

• H03M7/607

  Selection between different types of compressors · CPC title

• G06F16/00

  Information retrieval; Database structures therefor; File system structures therefor · CPC title