Data synchronization in a data analysis system

Patent metadata

Field	Value
Publication number	US-11704335-B2
Application number	US-202017097848-A
Country	US
Kind code	B2
Filing date	Nov 13, 2020
Priority date	Nov 13, 2020
Publication date	Jul 18, 2023
Grant date	Jul 18, 2023

Abstract

A computer implemented method comprises receiving at least one log record. A type of operation performed in a source database system may be determined based on the log record. Based on the type of operation, an update strategy may be selected from a set of pre-defined update strategies, wherein the update strategy specifies a sequence of replication operations to be performed in order to apply changes described in the at least one log record to a target database system.

First claim

The invention claimed is:

1. A computer implemented method for data synchronization in a data analysis system, the method comprising
receiving at least one log record, the log record describing an operation performed in a source database system of the data analysis system;
determining, based on the log record, a type of operation performed in the source database system;
based on the type of operation, selecting an update strategy from a set of pre-defined update strategies, the update strategy specifying a sequence of replication operations to be performed in order to apply changes described in the at least one log record to a target database system of the data analysis system; and
determining a class of a change record of the log record, wherein the class is determinable as being a regular class, a structural change class, and a utility action class;
wherein the update strategy uses:
a regular record add, delete, or update operation for the class determined to be a regular class;
a bulk load of the target database system with data originating from the source database system for the class determined to be a structural change class; and
a suspension operation for suspending modifications of a certain table of the target database system for the class determined to be a utility action class.

2. The method of claim 1 , wherein the at least one log record comprises a sequence of log records pertaining a specific table of the source database system, determining the type of operation being based on the sequence.

3. The method of claim 1 , wherein determining the type of the operation comprises comparing the number of operations indicated in the at least one log record with a pre-defined threshold and, based on the comparison result, determining the type of operation.

4. The method of claim 1 , wherein the method comprises applying the change described in the at least one log record according to the selected update strategy.

5. The method of claim 4 , wherein the method comprises:
detecting that applying the change failed;
in response to said detecting, selecting a secondary update strategy; and
applying the change based on the secondary update strategy.

6. The method of claim 1 , wherein the type of operation includes at least one of a regular change for adding, deleting or updating records of a table, a structural change for changing the structure of the table, or a utility action performed on the source database system.

7. A computer program product comprising a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configured to cause a processor to:
receive at least one log record, the log record describing an operation performed in a source database system of the data analysis system;
determine, based on the log record, a type of operation performed in the source database system;
based on the type of operation, select an update strategy from a set of pre-defined update strategies, the update strategy specifying a sequence of replication operations to be performed in order to apply changes described in the at least one log record to a target database system of the data analysis system; and
determine a class of a change record of the log record, wherein the class is determinable as being a regular class, a structural change class, and a utility action class;
wherein the update strategy uses:
a regular record add, delete, or update operation for the class determined to be a regular class;
a bulk load of the target database system with data originating from the source database system for the class determined to be a structural change class; and
a suspension operation for suspending modifications of a certain table of the target database system for the class determined to be a utility action class.

8. The computer program product of claim 7 , wherein the at least one log record comprises a sequence of log records pertaining a specific table of the source database system, determining the type of operation being based on the sequence.

9. The computer program product of claim 7 , wherein the computer-readable program code is configured to cause a processor to determine the type of the operation by comparing the number of operations indicated in the at least one log record with a pre-defined threshold and, based on the comparison result, determining the type of operation.

10. The computer program product of claim 7 , wherein the computer-readable program code is further configured to cause the processor to apply the change described in the at least one log record according to the selected update strategy.

11. The computer program product of claim 10 , wherein computer-readable program code is further configured to cause the processor to:
detect that applying the change failed;
in response to said detecting, select a secondary update strategy; and
apply the change based on the secondary update strategy.

12. The computer program product of claim 7 , wherein the type of operation includes at least one of a regular change for adding, deleting or updating records of a table, a structural change for changing the structure of the table, or a utility action performed on the source database system.

13. A computer system comprising:
a memory configured to store executable instructions; and
a processor communicatively coupled to the memory and configured to execute the executable instructions to:
receive at least one log record, the log record describing an operation performed in a source database system;
determine based on the log record a type of operation performed in the source database system;
based on the type of operation, selecting an update strategy from a set of pre-defined update strategies, the update strategy specifying a sequence of replication operations to be performed in order to apply changes described in the at least one log record to a target database system; and
determine a class of a change record of the log record, wherein the class is determinable as being a regular class, a structural change class, and a utility action class;
wherein the update strategy uses:
a regular record add, delete, or update operation for the class determined to be a regular class;
a bulk load of the target database system with data originating from the source database system for the class determined to be a structural change class; and
a suspension operation for suspending modifications of a certain table of the target database system for the class determined to be a utility action class.

14. The computer system of claim 13 , wherein the at least one log record comprises a sequence of log records pertaining a specific table of the source database system, determining the type of operation being based on the sequence.

15. The computer system of claim 13 , wherein the processor is configured to determine the type of the operation by comparing the number of operations indicated in the at least one log record with a pre-defined threshold and, based on the comparison result, determining the type of operation.

16. The computer system of claim 13 , wherein the processor is configured to apply the change described in the at least one log record according to the selected update strategy.

17. The computer system of claim 16 , wherein the processor is further configured to:
detect that applying the change failed;
in response to said detecting, select a secondary update strategy; and
apply the change based on the secondary update strategy.

18. The computer system of claim 13

Assignees

• IBM

Inventors

• Beier Felix
• Butterstein Dennis
• Perathoner-Tschaffler Sabine
• Lueck Einar

Classifications

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F16/2386 Primary

  Bulk updating operations (data conversion details G06F16/258) · CPC title

• G06F16/2358 Primary

  Change logging, detection, and notification (replication G06F16/27) · CPC title