Reordering of database records for improved compression

Patent metadata

Field	Value
Publication number	US-2015347087-A1
Application number	US-201414287589-A
Country	US
Kind code	A1
Filing date	May 27, 2014
Priority date	May 27, 2014
Publication date	Dec 3, 2015
Grant date	—

Abstract

According to embodiments of the present invention, apparatus, systems, methods and computer program products for sorting and compressing an unordered set of data records from a structured database are provided. Fields of the unordered set of data records are prioritized based on an impact of those fields to a compression scheme for column-oriented compression. The unordered set of data records are sorted based on the prioritized field(s) with a greatest impact on the performance metric. Data of the sorted data records are compressed according to a compression scheme. In some embodiments, prioritizing the fields may be based on an anticipated level of usage of data within those fields and/or a cost function associated with a performance metric as well as optimization of compression. A performance metric may include a faster computational time, reduced I/O computation, faster scan time, etc.

First claim

1 - 7 . (canceled)

8 . A system comprising:
a memory; and a processor configured to: prioritize fields of an unordered set of data records from a structured database based on an impact of those fields to a performance metric for accessing data stored in a column-oriented compressed database; sort the unordered set of data records based on the one or more prioritized field with a greatest impact on the performance metric; and compress data of the sorted data records according to a compression scheme.

9 . The system of claim 8 , wherein the processor is configured to prioritize the fields based on an anticipated level of usage of data within those fields.

10 . The system of claim 8 , wherein the processor is configured to prioritize the fields based on a performance metric as assessed by a cost function, the cost function including minimizing a total amount of time reading and computing on a column as well as the compressed size of data in the column.

11 . The system of claim 8 , wherein the processor is configured to sort the unordered set of data records based on a prioritized tuple of sort keys.

12 . The system of claim 8 , wherein the unordered set of data records is from a columnar database or a hybrid columnar database.

13 . The system of claim 8 , wherein the performance metric is determined based upon historical data.

14 . The system of claim 8 , wherein the processor is configured to:
update, based upon monitored performance metrics, relative priorities of the various fields to each other; decompress a set of ordered compressed records to obtain an unordered set of data records; sort the unordered set of data records based on the updated field priorities to obtain a second set of sorted data records; and compress data of the second set of sorted data records according to a compression scheme.

15 . A computer program product for providing information to a user, comprising a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, when executed by a processor, causes the processor to:
prioritize fields of an unordered set of data records from a structured database based on an impact of those fields to a performance metric for accessing data stored in a column-oriented compressed database; sort the unordered set of data records based on the one or more prioritized field with a greatest impact on the performance metric; and compress data of the sorted data records according to a compression scheme.

16 . The computer program product of claim 15 , the computer readable program code further configured to cause the processor to prioritize the fields based on an anticipated level of usage of data within those fields.

17 . The computer program product of claim 15 , the computer readable program code further configured to cause the processor to prioritize the fields based on a performance metric as assessed by a cost function, the cost function including minimizing a total amount of time reading and computing on a column as well as the compressed size of data in the column.

18 . The computer program product of claim 15 , the computer readable program code further configured to cause the processor to sort the unordered set of data records based on a prioritized tuple of sort keys.

19 . The computer program product of claim 15 , wherein the performance metric is determined based upon historical data.

20 . The computer program product of claim 15 , the computer readable program code further configured to cause the processor to:
update, based upon monitored performance metrics, relative priorities of the various fields to each other; decompress a set of ordered compressed records to obtain an unordered set of data records; sort the unordered set of data records based on the updated field priorities to obtain a second set of sorted data records; and compress data of the second set of sorted data records according to a compression scheme.

Assignees

• IBM

Inventors

• Dickie Garth A
• Keller Jeffrey M

Classifications

• G06F17/30469

  Physics · mapped topic

• G06F7/36 Primary

  Combined merging and sorting · CPC title

• G06F17/30595

  Physics · mapped topic

• G06F17/30315

  Physics · mapped topic

• G06F16/284

  Relational databases · CPC title