DML replication with logical log shipping

Patent metadata

Field	Value
Publication number	US-9720992-B2
Application number	US-201414302320-A
Country	US
Kind code	B2
Filing date	Jun 11, 2014
Priority date	Nov 22, 2013
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

Technologies are described for performing replication within a database environment. For example, replication of database data can be performed using data manipulation language (DML) statements and logical logs. A database node can execute a DML statement, create a logical log comprising the DML statement, and insert the logical log into a logical log queue. The logical log can be sent to one or more other database nodes for execution of the DML statement stored within the logical log. Logical logs can be grouped for sending to other nodes. Logical logs can be executed on a first node and one or more other nodes within the same transaction boundary. Execution of DML statements can be decoupled from sending, receiving, grouping, and/or processing of logical logs.

First claim

What is claimed is:

1. A method, implemented at least in part by a master node comprising a processing unit and memory, for performing replication within a database environment using data manipulation language (DML) statements and logical logs, the method comprising:
receiving a DML statement;
executing the DML statement at the master node;
creating a logical log comprising the DML statement;
inserting the logical log into a logical log queue; and
sending the logical log, retrieved from the logical log queue, to at least one slave node for execution of the DML statement at the at least one slave node;
wherein a first thread, executing separately from a second thread, performs logical log shipping, wherein the logical log shipping comprises at least one of: retrieving logical logs from the logical log queue, grouping, or sending to slave nodes; and
wherein the second thread performs the executing operation but does not perform the logical log shipping.

2. The method of claim 1 , wherein executing the DML statement at the master node comprises:
obtaining parameter values used by the DML statement; and
evaluating nondeterministic functions used by the DML statement to obtain nondeterministic values.

3. The method of claim 1 , the logical log further comprising:
a transaction identifier;
parameter values used by the DML statement; and
nondeterministic values used by the DML statement.

4. The method of claim 1 , wherein executing the DML statement at the master node is performed within a transaction boundary, and wherein the execution of the DML statement at the at least one slave node is performed within the transaction boundary.

5. The method of claim 4 , wherein execution of the DML statement at the master node and at the at least one slave node within the transaction boundary supports rollback of the DML statement at the master node and the at least one slave node.

6. The method of claim 1 , wherein execution of the DML statement at the master node is performed by a structured query language (SQL) executor thread, and wherein sending the logical log to the at least one slave node is performed by a replication handler thread.

7. The method of claim 1 , further comprising:
grouping the logical log with other logical logs in the logical log queue for sending to the at least one slave node.

8. The method of claim 7 , wherein the logical log and the other logical logs in the logical log queue are grouped based at least in part on which database tables are replicated at the at least one slave node.

9. A computer-readable storage medium storing computer-executable instructions for causing a computing device to execute a method for performing replication within a database environment using structured query language (SQL) statements and logical logs, the method comprising:
for each of a plurality of SQL statements:
receiving the SQL statement;
executing the SQL statement;
creating a logical log comprising the SQL statement; and
inserting the logical log into a logical log queue;

grouping the plurality of logical logs in the logical log queue into one or more logical log groups based at least in part upon database tables replicated by each of a plurality of slave nodes;
wherein first and second logical logs of the plurality of logical logs comprise respective first and second SQL statements directed to different respective first and second database tables replicated by a first slave node of the plurality of slave nodes, and the first and second logical logs are grouped together into a first logical log group of the one or more logical log groups; and
for each of the logical log groups, sending the logical log group to one or more slave nodes, of the plurality of slave nodes, that are associated with the logical log group for execution of the SQL statements in the logical log group.

10. The computer-readable storage medium of claim 9 wherein each logical log in the logical log queue that modifies the first database table is grouped together in the first logical log group and sent to each slave node that replaeicates the first database table.

11. The computer-readable storage medium of claim 9 wherein:
each logical log in the logical log queue that modifies the first database table is grouped together in the first logical log group;
the plurality of slave nodes comprises a first set of at least one slave node and a second set of at least one slave node;
each slave node of the first set is associated with the first logical group and replicates the first database table;
each slave node of the second set is not associated with the first logical group and does not replicate the first database table, and
the first logical log group is not sent to the at least one slave node of the second set.

12. The computer-readable storage medium of claim 9 wherein a first slave node replicates the first database table, wherein a second slave node replicates the second database table but does not replicate the first database table, wherein logical logs in the logical log queue that have SQL statements modifying the first database table are grouped together for sending to the first slave node but not to the second slave node, and wherein logical logs in the logical log queue that have SQL statements modifying the second database table are grouped together for sending to the second slave node.

13. The computer-readable storage medium of claim 9 , wherein a first thread, executing separately from a second thread, performs logical log shipping, wherein the logical log shipping comprises at least one of: retrieving logical logs from the logical log queue, grouping, or sending to slave nodes; and
wherein the second thread performs the executing operation but does not perform the logical log shipping.

14. A database environment comprising:
a master node; and
a plurality of slave nodes;
the database environment configured to perform operations for replicating database information using structured query language (SQL) statements and logical logs, the operations comprising:
at the master node:
receiving a SQL statement;
executing the SQL statement;
creating a logical log comprising the SQL statement;
inserting the logical log into a logical log queue;
grouping the logical log with other logical logs in the logical log queue for sending to at least one slave node of the plurality of slave nodes; and
sending the logical log from the logical log queue to the at least one slave node of the plurality of slave nodes for execution of the SQL statement at the at least one slave node; and

by the at least one slave node:
receiving the logical log; and
executing the DML SQL statement contained in the logical log;

wherein the logical log and the other logical logs in the logical log queue are grouped based at least in part on which database tables are replicated at the at least one slave node;
wherein first and second logical logs of a first group of logical logs comprise respective first and second SQL statements directed to different respective first and second database tables replicated by a first slave node of the at least one slave node.

15. The database environment of claim 14 , wherein executing the SQL statement at the master node comprises:
obtaining parameter values used by the SQL statement; and
evaluating nondeterministic functions used by the SQL statement to obtain nondeterministic values.

16. The database environment of claim 14 , the logical log further comprising:
a transaction identifier;
parameter values used by the SQL statement; and
nondeterministic val

Assignees

• Lee Joo Yeon
• Lee Juchang
• Kim Kyu Hwan
• Lee Chul Won
• Kwon Yong Sik
• Cha Sang Kyun
• Sap Se

Inventors

• Lee Joo Yeon
• Lee Juchang
• Kim Kyu Hwan
• Lee Chul Won
• Kwon Yong Sik
• Cha Sang Kyun

Classifications

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F17/30575 Primary

  Physics · mapped topic