Dynamic selection of synchronization update path

Patent metadata

Field	Value
Publication number	US-11397718-B2
Application number	US-202017016314-A
Country	US
Kind code	B2
Filing date	Sep 9, 2020
Priority date	Sep 9, 2020
Publication date	Jul 26, 2022
Grant date	Jul 26, 2022

Abstract

A method comprises receiving a stream of change log records from a source database system; generating change statistics based on a number of pending changes per table partition according to the change log records; estimating, based on performance statistics, a first amount of time for applying the pending changes to a target database system using an incremental update path; estimating, based on the performance statistics, a second amount of time for applying the pending changes to the target database using a bulk update path; dynamically selecting, based on comparison of the first amount of time with the second amount of time, one of the incremental update path and the bulk update path for applying the pending changes to the target database system; and applying the pending changes to the target database system using the selected update path.

First claim

What is claimed is:

1. A method comprising:
receiving a stream of change log records from a source database system;
generating change statistics based on a number of pending changes per table partition according to the change log records;
estimating, based on performance statistics, a first amount of time for applying the pending changes to a target database system using an incremental update path;
estimating, based on the performance statistics, a second amount of time for applying the pending changes to the target database using a bulk update path;
dynamically selecting, based on comparison of the first amount of time with the second amount of time, one of the incremental update path and the bulk update path for applying the pending changes to the target database system; and
applying the pending changes to the target database system using the selected update path.

2. The method of claim 1 , wherein each change log record comprises a log sequence number (LRSN), a timestamp, a table ID, a partition ID, and attribute changes for the corresponding record.

3. The method of claim 2 , wherein applying the pending changes comprises applying the pending changes via the bulk update path;
wherein the method further comprises:
discarding buffered changes for the incremental update path;
updating a value in the change statistics for a last applied log record with the LRSN of the last valid log record applied in the bulk update path; and
continuing log reading from the updated LRSN of the last applied log record.

4. The method of claim 1 , further comprising:
generating the performance statistics by monitoring application of changes to the target database system using the incremental update path and monitoring application of changes to the target database using the bulk update path.

5. The method of claim 1 , wherein dynamically selecting comprises dynamically selecting the incremental update path if the first amount of time is equal to or less than the second amount of time.

6. The method of claim 1 , wherein dynamically selecting comprises selecting the bulk update path if the second of amount of time is equal to or less than the first amount of time.

7. The method of claim 1 , further comprising:
for each change log record, identifying a table of a plurality of tables in the target database system to which the change applies based on data in the corresponding change log record; and
wherein dynamically selecting comprises dynamically selecting for each of the plurality of tables in the database, one of the incremental update path and the bulk update path; and
wherein applying the pending changes comprises applying the pending changes to the target database system using the corresponding selected update path for each of the plurality of tables.

8. A system comprising:
a source database system comprising a storage device configured to store a first dataset;
a target database system comprising a storage device configured to store a second dataset; and
a data synchronization system communicatively coupled to the source database system and the target database system, wherein the data synchronization system comprises a processor configured to:
receive a stream of change log records from the source database system;
generate change statistics based on a number of pending changes per table partition according to the change log records;
estimate, based on performance statistics, a first amount of time for applying the pending changes to the target database system using an incremental update path;
estimate, based on the performance statistics, a second amount of time for applying the pending changes to the target database using a bulk update path;
dynamically select, based on comparison of the first amount of time with the second amount of time, one of the incremental update path and the bulk update path for applying the pending changes to the target database system; and
apply the pending changes to the target database system using the selected update path.

9. The system of claim 8 , wherein each change log record comprises a log sequence number (LRSN), a timestamp, a table ID, a partition ID, and attribute changes for the corresponding record.

10. The system of claim 9 , wherein the processor is configured to apply the pending changes via the bulk update path; and
wherein the processor is further configured to:
discard buffered changes for the incremental update path;
update a value in the change statistics for a last applied log record with the LRSN of the last valid log record applied in the bulk update path; and
continue log reading from the updated LRSN of the last applied log record.

11. The system of claim 8 , wherein the processor is further configured to:
generate the performance statistics by monitoring application of changes to the target database system using the incremental update path and monitoring application of changes to the target database using the bulk update path.

12. The system of claim 8 , wherein the processor is configured to dynamically select the incremental update path if the first amount of time is equal to or less than the second amount of time.

13. The system of claim 8 , wherein the processor is configured to dynamically select the bulk update path if the second of amount of time is equal to or less than the first amount of time.

14. The system of claim 8 , wherein the processor is further configured to:
for each change log record, identify a table of a plurality of tables in the target database system to which the change applies based on data in the corresponding change log record; and
dynamically select for each of the plurality of tables in the database, one of the incremental update path and the bulk update path; and
apply the pending changes to the target database system using the corresponding selected update path for each of the plurality of tables.

15. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed by a processor, causes the processor to:
receive a stream of change log records from a source database system;
generate change statistics based on a number of pending changes per table partition according to the change log records;
estimate, based on performance statistics, a first amount of time for applying the pending changes to a target database system using an incremental update path;
estimate, based on the performance statistics, a second amount of time for applying the pending changes to the target database using a bulk update path;
dynamically select, based on comparison of the first amount of time with the second amount of time, one of the incremental update path and the bulk update path for applying the pending changes to the target database system; and
apply the pending changes to the target database system using the selected update path.

16. The computer program product of claim 15 , wherein each change log record comprises a log sequence number (LRSN), a timestamp, a table ID, a partition ID, and attribute changes for the corresponding record.

17. The computer program product of claim 16 , wherein the computer readable program is further configured to cause the processor to apply the pending changes via the bulk update path;
discard buffered changes for the incremental update path after applying the pending changes via the bulk update path;
update a value in the change statistics for a last applied log record with the LRSN of the last valid log record applied in the bu

Assignees

• IBM

Inventors

• Beier Felix
• Perathoner-Tschaffler Sabine
• Butterstein Dennis
• Lueck Einar

Classifications

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title

• G06F16/2322

  using timestamps · CPC title

• G06F16/235

  Update request formulation · CPC title

• G06F16/217 Primary

  Database tuning (G06F16/2282 takes precedence; database performance monitoring G06F11/3409) · CPC title

• G06F11/3409

  for performance assessment · CPC title