Database operations on a columnar table database

Patent metadata

Field	Value
Publication number	US-9418091-B2
Application number	US-201314033203-A
Country	US
Kind code	B2
Filing date	Sep 20, 2013
Priority date	Sep 6, 2013
Publication date	Aug 16, 2016
Grant date	Aug 16, 2016

Abstract

A computer system includes at least one processor and at least one memory operably coupled to the at least one processor. The memory includes a memory pool and a database partitioned into multiple fragments. Each of the fragments is allocated a block of memory from the memory pool and the fragments store compressed data in a columnar table format. A database operation is applied in a compressed format to the compressed data in at least one of the fragments.

First claim

What is claimed is:

1. A computer system, comprising:
at least one processor; and
at least one memory operably coupled to the at least one processor, the memory comprising:
a memory pool,
a database partitioned into a plurality of fragments, wherein each of the fragments is allocated a block of memory from the memory pool and the fragments store compressed data in a columnar table format, and
a change log that is configured to store uncompressed data,
wherein a database operation is applied in a compressed format to the compressed data in at least one of the fragments, the database operation including
an insert operation that causes inserting of new data in compressed format to one of the fragments, and
an update operation that causes:
locating a corresponding row of data in one of the fragments,
if a memory space in the fragment is sufficient to include one or more updated values in the data in compressed format, compressing the updated values and replacing the updated values in the data, and
if the memory space in the fragment is not sufficient to include the updated values in the data in compressed format, recording the updated values in the change log in an uncompressed format.

2. The computer system of claim 1 , wherein:
the insert operation causes appending of the new data in compressed format to a last row in one of the fragments.

3. The computer system of claim 1 , wherein:
the database operation includes a read operation, and
the read operation causes:
reading the compressed data from at least one of the fragments,
decompressing the compressed data,
reading the change log for associated data, and
combining the decompressed data from the fragment with the associated data from the change log.

4. The computer system of claim 1 , wherein the database is configured to compress the data in the change log and merge the compressed data from the change log with the compressed data in the fragments.

5. The computer system of claim 1 , wherein the database comprises a compression engine that is configured to compress data using one of a plurality of compression schemes.

6. The computer system of claim 5 , wherein the compression engine uses a dictionary encoding scheme to compress the data stored in the fragments.

7. The computer system of claim 1 , wherein the database is an in-memory database.

8. A method comprising:
partitioning a database into a plurality of fragments, wherein each of the fragments is allocated a block of memory from a memory pool;
storing compressed data in each of the fragments in a columnar table format;
applying a database operation in a compressed format to the compressed data in at least one of the fragments, wherein the database operation includes an insert operation and an update operation;
responsive to the insert operation, inserting new data in compressed format to one of the fragments; and
responsive to the update operation:
locating a corresponding row of data in one of the fragments,
if a memory space in the fragment is sufficient to include one or more updated values in the data in compressed format, compressing the updated values and replacing the updated values in the data, and
if the memory space in the fragment is not sufficient to include the updated values in the data in compressed format, recording the updated values in a change log in an uncompressed format, wherein the change log is configured to store uncompressed data.

9. The method of claim 8 , wherein inserting the new data further comprises:
responsive to the insert operation, appending the new data in compressed format to a last row in one of the fragments.

10. The method of claim 8 , wherein the database operation includes a read operation and the method further comprises responsive to the read operation:
reading the compressed data from at least one of the fragments,
decompressing the compressed data,
reading the change log for associated data, wherein the change log is configured to store uncompressed data, and
combining the decompressed data from the fragment with the associated data from the change log.

11. The method of claim 8 , further comprising:
compressing the data in the change log; and
merging the compressed data from the change log with the compressed data in the fragments.

12. The method of claim 8 , further comprising compressing data using one of a plurality of compression schemes.

13. The method of claim 12 , wherein the compression schemes include a dictionary encoding scheme.

14. A computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed, are configured to:
partition a database into a plurality of fragments, wherein each of the fragments is allocated a block of memory from a memory pool;
store compressed data in each of the fragments in a columnar table format;
apply a database operation in a compressed format to the compressed data in at least one of the fragments, wherein the database operation includes an insert operation and an update operation;
responsive to the insert operation, insert new data in compressed format to one of the fragments; and
responsive to the update operation:
locate a corresponding row of data in one of the fragments,
if a memory space in the fragment is sufficient to include one or more updated values in the data in compressed format, compress the updated values and replace the updated values in the data, and
if the memory space in the fragment is not sufficient to include the updated values in the data in compressed format, record the updated values in a change log in an uncompressed format, wherein the change log is configured to store uncompressed data.

15. The computer program product of claim 14 , wherein the instructions, when executed, are further configured to:
responsive to the insert operation, append the new data in compressed format to a last row in one of the fragments.

16. The computer program product of claim 14 , wherein the database operation includes a read operation and the instructions, when executed, are further configured to, responsive to the read operation:
read the compressed data from at least one of the fragments,
decompress the compressed data,
read the change log for associated data, wherein the change log is configured to store uncompressed data, and
combine the decompressed data from the fragment with the associated data from the change log.

17. The computer program product of claim 14 wherein, the instructions, when executed, are further configured to:
compress the data in the change log; and
merge the compressed data from the change log with the compressed data in the fragments.

Assignees

• Li Wen-Syan
• Dong Bin
• Wei Zheng Long
• Chen Yingyu
• Shen Yongyuan
• Sap Se

Inventors

• Li Wen-Syan
• Dong Bin
• Wei Zheng Long
• Chen Yingyu
• Shen Yongyuan

Classifications

• G06F16/221 Primary

  Column-oriented storage; Management thereof · CPC title

• G06F17/30315 Primary

  Physics · mapped topic