Managing transaction size during database replication

Patent metadata

Field	Value
Publication number	US-11841876-B2
Application number	US-202117482406-A
Country	US
Kind code	B2
Filing date	Sep 22, 2021
Priority date	Sep 22, 2021
Publication date	Dec 12, 2023
Grant date	Dec 12, 2023

Abstract

Techniques are described with regard to managing transaction size during a database replication process. An associated computer-implemented method includes identifying each of at least one transaction associated with at least one source database stored in a replication capture memory, identifying at least one potential excessive memory transaction by applying, via at least one machine learning pattern detection model, pattern detection to each of the at least one transaction stored in the replication capture memory, and constructing at least one compact data unit associated with each of the at least one potential excessive memory transaction. The computer-implemented method further includes facilitating compact data unit transmission to a replication apply server system in order to replicate the at least one compact data unit associated with each of the at least one potential excessive memory transaction to at least one target database via a replication apply server application.

First claim

What is claimed is:

1. A computer-implemented method of managing transaction size during a database replication process, the method comprising:
identifying each of at least one transaction associated with at least one source database stored in a replication capture memory;
identifying at least one potential excessive memory transaction by applying, via at least one machine learning pattern detection model, pattern detection to each of the at least one transaction stored in the replication capture memory;
constructing at least one compact data unit associated with each of the at least one potential excessive memory transaction by determining a transaction split pattern for the potential excessive memory transaction and splitting the potential excessive memory transaction based upon the transaction split pattern; and
facilitating compact data unit transmission to a replication apply server system in order to replicate the at least one compact data unit associated with each of the at least one potential excessive memory transaction to at least one target database via a replication apply server application.

2. The computer-implemented method of claim 1 , wherein identifying the at least one potential excessive memory transaction comprises:
predicting, via a future operation prediction model among the at least one machine learning pattern detection model, a complete view of each of the at least one transaction stored in the replication capture memory, the complete view including at least one predicted operation;
determining, via a memory impact model among the at least one machine learning pattern detection model, a predicted incremental memory impact value for each of the at least one transaction stored in the replication capture memory based upon the complete view; and
identifying any transaction among the at least one transaction stored in the replication capture memory as a potential excessive memory transaction responsive to determining that the predicted incremental memory impact value for the transaction exceeds an incremental memory impact threshold.

3. The computer-implemented method of claim 1 , wherein the potential excessive memory transaction is split based upon the transaction split pattern only responsive to determining that a data size of the potential excessive memory transaction exceeds a predetermined data size threshold.

4. The computer-implemented method of claim 1 , wherein determining the transaction split pattern for the potential excessive memory transaction comprises:
calculating similarity between at least one pattern associated with the potential excessive memory transaction and a plurality of historical transaction split patterns; and
selecting as the transaction split pattern a pattern among the plurality of historical transaction split patterns having a highest calculated similarity with the at least one pattern associated with the potential excessive memory transaction.

5. The computer-implemented method of claim 1 , wherein replicating the at least one compact data unit associated with each of the at least one potential excessive memory transaction to the at least one target database via the replication apply server application comprises:
creating a queue in a replication apply memory of the replication apply server system to link all compact data units among the at least one compact data unit associated with the potential excessive memory transaction;
completing replication apply processing for each compact data unit among the at least one compact data unit associated with the potential excessive memory transaction; and
committing the potential excessive memory transaction upon applying to the at least one target database all compact data units among the at least one compact data unit associated with the potential excessive memory transaction.

6. The computer-implemented method of claim 5 , wherein completing replication apply processing for each compact data unit among the at least one compact data unit associated with the potential excessive memory transaction comprises:
addressing any dependency associated with the compact data unit;
sequentially applying the compact data unit to the at least one target database; and
logging the sequential application of the compact data unit in a potential excessive memory transaction apply log including data associated with compact data units applied to the at least one target database but yet to be committed.

7. A computer program product comprising a computer readable storage medium having program instructions embodied therewith for managing transaction size during a database replication process, the program instructions executable by a computing device to cause the computing device to:
identify each of at least one transaction associated with at least one source database stored in a replication capture memory;
identify at least one potential excessive memory transaction by applying, via at least one machine learning pattern detection model, pattern detection to each of the at least one transaction stored in the replication capture memory;
construct at least one compact data unit associated with each of the at least one potential excessive memory transaction by determining a transaction split pattern for the potential excessive memory transaction and splitting the potential excessive memory transaction based upon the transaction split pattern; and
facilitate compact data unit transmission to a replication apply server system in order to replicate the at least one compact data unit associated with each of the at least one potential excessive memory transaction to at least one target database via a replication apply server application.

8. The computer program product of claim 7 , wherein identifying the at least one potential excessive memory transaction comprises:
predicting, via a future operation prediction model among the at least one machine learning pattern detection model, a complete view of each of the at least one transaction stored in the replication capture memory, the complete view including at least one predicted operation;
determining, via a memory impact model among the at least one machine learning pattern detection model, a predicted incremental memory impact value for each of the at least one transaction stored in the replication capture memory based upon the complete view; and
identifying any transaction among the at least one transaction stored in the replication capture memory as a potential excessive memory transaction responsive to determining that the predicted incremental memory impact value for the transaction exceeds an incremental memory impact threshold.

9. The computer program product of claim 7 , wherein the potential excessive memory transaction is split based upon the transaction split pattern only responsive to determining that a data size of the potential excessive memory transaction exceeds a predetermined data size threshold.

10. The computer program product of claim 7 , wherein determining the transaction split pattern for the potential excessive memory transaction comprises:
calculating similarity between at least one pattern associated with the potential excessive memory transaction and a plurality of historical transaction split patterns; and
selecting as the transaction split pattern a pattern among the plurality of historical transaction split patterns having a highest calculated similarity with the at least one pattern associated with the potential excessive memory transaction.

11. The computer program product of claim 7 , wherein replicating the at least one compact data unit associated with each of the at least one potential excessive memory transaction to the at

Assignees

• IBM

Inventors

• Zhang he fang
• Bo Xue
• Li De Song
• Zhou Wen Kui
• Xia Yin
• Chen Dong

Classifications

• G06F16/273 Primary

  Asynchronous replication or reconciliation · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F9/466

  Transaction processing · CPC title