Data synchronization between a source database system and target database system

Patent metadata

Field	Value
Publication number	US-11893041-B2
Application number	US-202016830696-A
Country	US
Kind code	B2
Filing date	Mar 26, 2020
Priority date	May 15, 2019
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

The present disclosure relates to a method for data synchronization between a source database system and target database system, wherein execution of a database transaction of the source database system is complete if a processing step followed by an application step of the database transaction is performed. For each identified database transaction, a processing step and/or application step may be performed.

First claim

What is claimed is:

1. A computer-implemented method for data synchronization between a source database system and a target database system, wherein execution of a database transaction of the source database system is considered to be complete if a processing step followed by an application step of the database transaction is performed, the method comprising:
identifying in a time interval one or more database transactions of the source database system via a transaction log of the source database system which comprises log records resulting from the one or more database transactions;
tagging each of the one or more database transactions of the transaction log as a committed or an uncommitted database transaction;
sending the transaction log to the target database system for implementing changes of the one or more database transactions, wherein prior to the sending log records of the one or more database transactions are transformed according to a data record structure of the target database system, and wherein the transformations comprise changing a format of the log records;
for each transaction of the identified database transactions, determining, in parallel, whether a transaction was completed based on a tag of the transaction, wherein determining whether the transaction was completed further comprises:
in response to determining that the transaction is not completed, performing, by the target database system, the processing step of the transaction if a duration time of the transaction exceeds a duration threshold, wherein performing the processing step comprises invoking a respective additional thread of a transaction process of the target database system;
in response to determining that the transaction is completed and the processing step of the transaction was not previously executed, performing, by the target database system, the processing step and the application step of the transaction, wherein performing the processing step and the application step comprises invoking a main thread of the transaction process; and
in response to determining that the transaction is completed and the processing step of the transaction was previously executed, performing, by the target database, the application step of the transaction, wherein performing the application step comprises invoking another respective additional thread of the transaction process of the target database system; and

iteratively performing the identifying, the tagging, and the determining, wherein in each iteration the time interval is a time interval subsequent to a time interval of a preceding iteration, and wherein a length of a subsequent time interval is different from a length of a preceding time interval.

2. The computer-implemented method of claim 1 , wherein the processing step comprises beginning the transaction and executing a set of data manipulations of the transaction, and wherein the application step comprises committing the transaction and ending the transaction, and wherein the transaction process executes transactions in the target database system.

3. The computer-implemented method of claim 1 , further comprising:
performing, in accordance with a batch processing having scheduling time intervals, steps of identifying in a time interval one or more database transactions of the source database system and determining whether the transaction was completed.

4. The computer-implemented method of claim 1 , wherein a completion of each database transaction for a respective database in the source database system results in a specific content of the respective database, and wherein steps performing, by the target database system, the processing step and/or the application step of the transaction are performed only if a resulting content of each database affected by the transaction in the target database system is one of the specific contents.

5. The computer-implemented method of claim 1 , wherein the source database system is configured to perform the processing and application steps for different transactions in a given order taking into account the dependencies between data transactions to prevent data inconsistency, wherein determining whether the transaction was completed further comprises: determining if at least one another transaction of the identified database transactions is dependent on the transaction, and if so performing the steps, in response to determining that the transaction is not completed, performing, by the target database system, the processing step of the transaction and in response to determining that the transaction is completed and the processing step of the transaction was not previously executed, performing, by the target database system, the processing step and the application step of the transaction, if the transaction occurred before the application step of each of the at least one other transaction, otherwise performing the steps, in response to determining that the transaction is not completed, performing, by the target database system, the processing step of the transaction and in response to determining that the transaction is completed and the processing step of the transaction was not previously executed, performing, by the target database system, the processing step and the application step of the transaction, after the step of determining whether the transaction was completed is executed for the other transactions.

6. The computer-implemented method of claim 1 , wherein the source database system comprises a transaction log for logging log records resulting from the database transactions of the source database system and a log reader for reading the transaction log, and wherein the source database system buffers read data for enabling change replication to the target database system, and wherein the step, in response to determining that the transaction is not completed, performing, by the target database system, the processing step of the transaction, further comprises controlling the log reader to remove log records resulting from the processing step of said transaction.

7. The computer-implemented method of claim 1 , wherein the source database system comprises a transaction log for logging log records resulting from the database transactions, and wherein identifying the database transactions is performed using the transaction log.

8. The computer-implemented method of claim 1 , wherein the step, in response to determining that the transaction is not completed, performing, by the target database system, the processing step of the transaction, is only performed if the duration and/or size of the transaction fulfil a predefined condition.

9. The computer-implemented method of claim 1 , wherein determining that the transaction is completed comprises determining that the transaction is tagged as a committed data transaction, and wherein determining that said transaction is not completed comprises determining that said transaction is tagged as an uncommitted data transaction.

10. The computer-implemented method of claim 1 , further comprising:
in response to determining that said transaction is cancelled, determining if the step, performing, by the target database system, the processing step of the transaction is performed for the transaction while the step, performing, by the target database, the application step of the transaction, has not been performed and if so causing the target database system to roll back the result of the processing step of the transaction.

11. A computer program product for data synchronization between a source database system and a target database system, wherein execution of a database transaction of the source database system is considered to be complete if a processi

Assignees

• IBM

Inventors

• Stolze Knut
• Dimov Vassil Radkov
• Michel Christian
• Martin Daniel

Classifications

• G06F16/275 Primary

  Synchronous replication · CPC title

• G06F16/178 Primary

  Techniques for file synchronisation in file systems · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title