Database recovery and optimization with batch processing

Patent metadata

Field	Value
Publication number	US-10824519-B2
Application number	US-201815953856-A
Country	US
Kind code	B2
Filing date	Apr 16, 2018
Priority date	Apr 16, 2018
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

Disclosed herein are system, method, and computer program product embodiments for a database recovery and optimization with batch processing system. An embodiment operates by retrieving a database log that includes a plurality of operations for modifying data of a database stored across a plurality of tables. From the database log, a plurality of consecutive insert operations for inserting data into the database are identified. The consecutive insert operations are sorted by table. The sorted insert operations are grouped into a batch message. The batch message is transmit to the database for replay. An acknowledgement is received that the replay has completed.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
retrieving a database log that includes a plurality of operations that modified either data or schema across a plurality of tables of a database and that were executed but not persisted prior to the retrieving;
sorting the consecutive insert operations into a plurality of groups of operations by both an operation type and the table of the database on which a particular operation was performed, wherein the log includes a plurality of different types of operation, and each group is executable in parallel on the database;
transmitting a batch message including the plurality of groups of operations to the database for replay of the sorted operations based on the groups; and
receiving an acknowledgement that the replay has completed.

2. The method of claim 1 , wherein the types of operations include both insert and non-insert operations, and wherein the sorting sorts the insert operations prior to the non-insert operations.

3. The method of claim 2 , wherein the non-insert operations comprise delete operations.

4. The method of claim 1 , wherein at least one of the plurality of operations comprises a schema change operation, wherein a schema of one of the plurality of tables is modified by the schema change operation.

5. The method of claim 4 , further comprising:
transmitting the schema change operation;
receiving an acknowledgement that the schema change operation has completed;
grouping one or more insert operations subsequent to the schema change operation into a second batch message; and
transmitting the second batch message.

6. The method of claim 1 , wherein the grouping comprises combining a column of a first insert operation with a column of a second insert operation.

7. The method of claim 1 , wherein the retrieving comprises:
determining that a database failure event associated with the database has occurred, wherein the database log comprises operations that were not persisted to the database prior to the failure event.

8. The method of claim 1 , wherein the grouping comprises:
grouping a first portion of the sorted insert operations into a first batch message corresponding to a first one of the plurality of tables; and
grouping a second portion of the sorted insert operations into a second batch message corresponding to a second one of the plurality of tables.

9. The method of claim 8 , wherein the receiving comprises:
receiving a second acknowledgement that the second batch message was processed by a second processor prior to receiving a first acknowledgement that the first batch message was processed by a first processor.

10. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
retrieve a database log that includes a plurality of operations that modified either data or schema across a plurality of tables of a database and that were executed but not persisted prior to the retrieving for modifying data of a database stored across a plurality of tables;
sort the consecutive insert operations into a plurality of groups of operations by both an operation type and the table of the database on which a particular operation was performed, wherein the log includes a plurality of different types of operation, and each group is executable in parallel on the database;
transmit a batch message including the plurality of groups of operations to the database for replay of the sorted operations based on the groups; and
receive an acknowledgement that the replay has completed.

11. The system of claim 10 , wherein the types of operations include both insert and non-insert operations, and wherein the processors sorts the insert operations prior to the non-insert operations.

12. The system of claim 11 , wherein the non-insert operations comprise delete operations.

13. The system of claim 10 , wherein at least one of the plurality of operations comprises a schema change operation, wherein a schema of one of the plurality of tables is modified by the schema change operation.

14. The system of claim 13 , wherein the processor is further configured to:
transmit the schema change operation;
receive an acknowledgement that the schema change operation has completed;
group one or more insert operations subsequent to the schema change operation into a second batch message; and
transmit the second batch message.

15. The system of claim 10 , wherein the grouping comprises combining a column of a first insert operation with a column of a second insert operation.

16. The system of claim 10 , wherein the processor that retrieves is configured to:
determine that a database failure event associated with the database has occurred, wherein the database log comprises operations that were not persisted to the database prior to the failure event.

17. The system of claim 10 , wherein the processor that groups is configured to:
group a first portion of the sorted insert operations into a first batch message corresponding to a first one of the plurality of tables; and
group a second portion of the sorted insert operations into a second batch message corresponding to a second one of the plurality of tables.

18. The system of claim 17 , wherein the processor that receives is configured to:
receive a second acknowledgement that the second batch message was processed by a second processor prior to receiving a first acknowledgement that the first batch message was processed by a first processor.

19. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
retrieving a database log that includes a plurality of operations that modified either data or schema across a plurality of tables of a database and that were executed but not persisted prior to the retrieving;
sorting the operations into a plurality of groups of operations by both an operation type and the table of the database on which a particular operation was performed, wherein the log includes a plurality of different types of operation, and each group is executable in parallel on the database;
transmitting a batch message including the plurality of groups of operations to the database for replay of the sorted operations based on the groups; and
receiving an acknowledgement that the replay has completed.

20. The method of claim 1 , wherein the retrieving is performed subsequent to a crash or reboot operation of one or more devices associated with the database.

Assignees

• Sap Se

Inventors

• Heidel Martin
• Liu Xin
• Roterring Christoph
• Chen Shiping
• Kandiyanallur Vivek
• Kottler Stephan
• Schmidt Joern

Classifications

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title

• G06F2201/80

  Database-specific techniques · CPC title