Transformation functions for compression and decompression of data in computing environments and systems

Patent metadata

Field	Value
Publication number	US-9558251-B2
Application number	US-201213478284-A
Country	US
Kind code	B2
Filing date	May 23, 2012
Priority date	Apr 3, 2012
Publication date	Jan 31, 2017
Grant date	Jan 31, 2017

Abstract

One or more transformation functions can be used in connection or together with one or more compression/decompression techniques. A transformation function can transform data (e.g., a data object) into a form more suitable for compression and/or decompression. As a result, data can be compressed and/or decompressed more effectively. In addition, multiple data objects can be associated with various transformation functions and/or compression/decompression techniques. As a result, different approaches can be taken with respect to compression and decompression of data objects in an effort to find an optimum approach for compression of data objects that may vary significantly from each other and change over time. It will be appreciated that the objects can be associated with transformation functions in a dynamic manner to accommodate changes to data. Also, an extendible and/or extensible system can allow for growth and adaption of new data in forms not currently present or expected.

First claim

What is claimed is:

1. A computer-implemented method, implemented at least partly on a device, for facilitating at least one of compression and decompression of data objects associated with a computing environment or system, the computer-implemented method comprising:
associating, by the device, at least one data transformation function to at least one of the data objects, wherein the data transformation function is designated to transform the at least one data object from a first form received as input to a second form as output, wherein the data transformation function includes at least one tumbling transposing function, and wherein the at least one data transformation function is further designated to be applied to the at least one data object in connection with application of at least one of: (i) before one or more compression techniques designated to compress the at least one data object are applied, and (ii) after one or more decompression techniques designated to decompress a compressed version of the at least one data object are applied; and
wherein the at least one data object is represented as one or more N×K byte sized partitions of monotonically increasing fixed-length values of size K, wherein N and K are positive integers greater than one;
wherein the at least one data transformation function includes a tumbling N×K byte matrix transpose function that can be applied to the one or more N×K byte sized partitions;
wherein the at least one data transformation function also includes a tumbling K×N byte matrix transpose function as the inverse function of the tumbling N×K byte matrix transpose function;
wherein the tumbling N×K byte matrix transpose function is designated to be used with a dictionary-based compression technique, and
wherein the K×N byte matrix transpose function is designated to be used with a dictionary-based decompression technique.

2. The computer-implemented method of claim 1 ,
wherein the at least one data transformation function includes a pre-compression transformation function designated to be applied to the at least one data object to obtain at least one transformed data object before the at least one transformed data object is compressed to obtain at least one compressed version of the at least one data object, and
wherein the computer-implemented method further comprises:

associating the at least one pre-compression transformation function that modifies the arrangement of the data for the compression of the data and prior to the compression of the data, to the at least one data object and a compression technique designated to compress the at least one transformed data object to obtain at least one compressed version of the at least one data object.

3. The computer-implemented method of claim 2 ,
wherein the at least one data transformation function includes a post-decompression transformation function designated to be applied to a decompressed version of the at least one compressed version of the at least one data object; and
wherein the computer-implemented method further comprises:

associating the post-decompression transformation function to the at least one data object and a decompression technique designated to decompress the at least one compressed version of the at least one data object to obtain at least one decompressed version of the at least one data object before the post-decompression transformation function is applied to the at least one decompressed version to obtain the at least one data object.

4. The computer-implemented method of claim 1 , wherein the computer-implemented method further comprises:
associating the at least one data object and the at least data transformation function also to at least one of: (a) a compression technique to be applied to the at least one database object and (b) a decompression technique to be applied to a compressed version of the at least one database object.

5. The computer-implemented method of claim 1 , wherein the computer-implemented method further comprises:
receiving at least one of: (i) at least one data object (ii) a compression technique, and (iii) a decompression technique; and
determining the least one transformation function for at least one of: (i) the at least one data object (ii) the compression technique, and (iii) the decompression technique.

6. The computer-implemented method of claim 1 , wherein the computer-implemented method further comprises:
receiving a first data object of the data objects and a compression technique to be applied to the first data object to compress the first data object; and
determining, based on one or more characteristics of the first data object, a transformation function to be applied to the first data object prior to using the compression technique to compress the first data object.

7. The computer-implemented method of claim 6 , wherein the computer-implemented further comprises:
examining the first data object to determine the one or more characteristics of the first data object.

8. The computer-implemented method of claim 1 , wherein the at least one data object includes one or more of the following:
a data portion or segment, a class or type of data, and a field.

9. The computer-implemented method of claim 1 , wherein the data transformation function is a transposing function.

10. The computer-implemented method of claim 1 ,
wherein K is eight (8) or a multiple of eight (8), and
wherein N is thirty two (32) Kilo Bytes (KB) or a multiple of 32 KB.

11. The computer-implemented method of claim 10 , wherein the compression and decompression techniques are based on LZ77, LZ78 technique.

12. The computer-implemented method of claim 11 , wherein the at least one data object includes a columnar offset for a database.

13. The computer-implemented method of claim 1 , wherein the associating associates a first database object of the data objects to a first transformation function and also associates a second database object of the data objects, different than the first data object, to a second transformation function which is different than the first transformation function.

14. A system that includes one or more processors operable to associate at least one data transformation function to at least one data object, wherein the data transformation function is designated to transform the at least one data object from a first form received as input to a second form as output, wherein the data transformation function includes at least one tumbling transposing function, and wherein the at least one data transformation function is further designated to be applied to the at least one data object in connection with application of at least one of: (i) before one or more compression techniques designated to compress the at least one data object, and (ii) after one or more decompression techniques designated to decompress a compressed version of the at least one data object;
wherein the at least one data object is represented as one or more N×K byte sized partitions of monotonically increasing fixed-length values of size K, wherein N and K are positive integers greater than one;
wherein the at least one data transformation function includes a tumbling N×K byte matrix transpose function that can be applied to the one or more N×K byte sized partitions;
wherein the at least one data transformation function also includes a tumbling K×N byte matrix transpose function as the inverse function of the tumbling N×K byte matrix transpose function;
wherein the tumbling N×K byte matrix transpose function is also designated to be used with a dictionary-based compression technique, and
wherein the K×N byte

Assignees

• Simmen David
• Hovsepian Shant
• Davis Jeffrey
• Teradata Us Inc

Inventors

• Simmen David
• Hovsepian Shant
• Davis Jeffrey

Classifications

• G06F16/278

  Data partitioning, e.g. horizontal or vertical partitioning · CPC title

• H03M7/30

  Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction · CPC title

• H03M7/702

  Software · CPC title

• H03M7/70

  Type of the data to be coded, other than image and sound · CPC title

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title