Maintaining in-memory database consistency by parallelizing persistent data and log entries

Patent metadata

Field	Value
Publication number	US-9858310-B2
Application number	US-201514853260-A
Country	US
Kind code	B2
Filing date	Sep 14, 2015
Priority date	Sep 14, 2015
Publication date	Jan 2, 2018
Grant date	Jan 2, 2018

Abstract

A database operation is initiated in an in-memory database that stores persistent data in a secondary storage and has at least one associated log. The database operation involves data having a size above a pre-defined threshold. Thereafter, the data is split into a plurality of chunks and the database operation is executed by parallelizing each chunk over a plurality of hierarchically arranged threads. Related systems, apparatus, methods and computer program products are also described.

First claim

What is claimed is:

1. A method for implementation by one or more data processors forming part of at least one computing system, the method comprising:
initiating a database operation in an in-memory database that stores persistent data in a secondary storage and has at least one associated log, the database operation involving data having a size above a pre-defined threshold;
splitting the data into a plurality of chunks; and
executing the database operation by parallelizing each chunk over a plurality of hierarchically arranged threads such that, for each chunk:
a parent thread causes each of a plurality of child threads to execute a corresponding consistent change;
the child threads each notify the parent thread when they have started their corresponding consistent change;
the parent thread starts its own consistent change upon receiving the notifications from all of the child threads that they have started their corresponding consistent change;
the parent thread, as part of its consistent change, then writes visibility information and undo and redo log entries in the log;
the child threads each notify the parent thread when they have completed their corresponding consistent change; and
the parent thread closes its consistent change upon receiving the notifications from all of the child threads that they have completed their corresponding consistent change.

2. The method of claim 1 , wherein the in-memory database is a column-oriented in-memory database that stores data tables as sections of columns of data rather than as rows of data.

3. The method of claim 1 , wherein there is one child thread per column of data being affected by the database operation.

4. The method of claim 1 , wherein there are less than one child thread per column of data being affected by the database operation.

5. The method of claim 1 , wherein the database operation is a data manipulation language (DML) operation to retrieve and/or manipulate data stored within the database.

6. The method of claim 5 , wherein the database operation is selected from a group consisting of: select, insert, update, and delete.

7. The method of claim 1 , wherein the pre-defined threshold is based on a number of cells implicated by the database operation.

8. A system comprising:
an in-memory database comprising memory and at least one data processor, the memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
initiating a database operation in the in-memory database that stores persistent data in a secondary storage and has at least one associated log, the database operation involving data having a size above a pre-defined threshold;
splitting the data into a plurality of chunks; and

executing the database operation by parallelizing each chunk over a plurality of hierarchically arranged threads.

9. The system of claim 8 , wherein, as part of the execution of the database operation, for each chunk:
a parent thread causes each of a plurality of child threads to execute a corresponding consistent change;
the child threads each notify the parent thread when they have started their corresponding consistent change;
the parent thread starts its own consistent change upon receiving the notifications from all of the child threads that they have started their corresponding consistent change;
the parent thread, as part of its consistent change, then writes visibility information and undo and redo log entries in the log;
the child threads each notify the parent thread when they have completed their corresponding consistent change; and
the parent thread closes its consistent change upon receiving the notifications from all of the child threads that they have completed their corresponding consistent change.

10. The system of claim 8 , wherein the in-memory database is a column-oriented in-memory database that stores data tables as sections of columns of data rather than as rows of data.

11. The system of claim 8 , wherein there is one child thread per column of data being affected by the database operation.

12. The system of claim 8 , wherein there are less than one child thread per column of data being affected by the database operation.

13. The system of claim 8 , wherein the database operation is a data manipulation language (DML) operation to retrieve and/or manipulate data stored within the database.

14. The system of claim 8 , wherein the pre-defined threshold is based on a number of cells implicated by the database operation.

15. A non-transitory computer program product storing instructions which, when executed by at least one data processor forming part of at least one computing system, result in operations comprising:
initiating a database operation in an in-memory database that stores persistent data in a secondary storage and has at least one associated log, the database operation involving data having a size above a pre-defined threshold;
splitting the data into a plurality of chunks; and
executing the database operation by parallelizing each chunk over a plurality of hierarchically arranged threads.

16. The computer program product of claim 15 , wherein, as part of the execution of the database operation, for each chunk:
a parent thread causes each of a plurality of child threads to execute a corresponding consistent change;
the child threads each notify the parent thread when they have started their corresponding consistent change;
the parent thread starts its own consistent change upon receiving the notifications from all of the child threads that they have started their corresponding consistent change;
the parent thread, as part of its consistent change, then writes visibility information and undo and redo log entries in the log;
the child threads each notify the parent thread when they have completed their corresponding consistent change; and
the parent thread closes its consistent change upon receiving the notifications from all of the child threads that they have completed their corresponding consistent change.

17. The computer program product of claim 16 , wherein the in-memory database is a column-oriented in-memory database that stores data tables as sections of columns of data rather than as rows of data.

18. The computer program product of claim 17 , wherein there is one child thread per column of data being affected by the database operation.

19. The computer program product of claim 17 , wherein there are less than one child thread per column of data being affected by the database operation.

20. The computer program product of claim 16 , wherein the database operation is a data manipulation language (DML) operation to retrieve and/or manipulate data stored within the database.

Assignees

• Sap Se

Inventors

• Heidel Martin
• Schreter Ivan
• Schmidt Joern

Classifications

• G06F16/2358 Primary

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title

• G06F17/30371

  Physics · mapped topic

• G06F17/30368 Primary

  Physics · mapped topic