Log-shipping data replication with early log record fetching

Patent metadata

Field	Value
Publication number	US-9864772-B2
Application number	US-89564410-A
Country	US
Kind code	B2
Filing date	Sep 30, 2010
Priority date	Sep 30, 2010
Publication date	Jan 9, 2018
Grant date	Jan 9, 2018

Abstract

Methods, systems and program products for log-shipping data replication from a primary system to a communicatively-coupled standby system. Embodiments of the invention may receive transactional log data at a standby system, from the primary system, and before the transactional log data is written to storage on the primary system. Embodiments may then receive a notification from the primary system indicating that the corresponding log data was written to storage on the primary system, and responsive to receiving the notification, may process the received transactional log data.

First claim

What is claimed is:

1. A computer-implemented method for data replication, comprising:
allocating, to a first thread on a primary system communicatively coupled to a standby system, a first buffer portion of a transaction log buffer for a first transaction of a plurality of transactions, wherein the transaction log buffer comprises a contiguous area of memory configured to store log data related to the plurality of transactions, wherein the first buffer portion is fully formed when filled with log data related to the first transaction and to be written to storage on the primary system;
allocating, to a second thread on the primary system, a second buffer portion of the transaction log buffer for a second transaction of the plurality of transactions, wherein the second buffer portion is fully formed when filled with log data related to the second transaction and to be written to storage on the primary system;
upon determining that the first buffer portion is fully formed, before the second buffer portion is fully formed, and before the log data in the first portion is written to the storage on the primary system, transmitting the log data in the first buffer portion to the standby system, from the primary system; and
upon writing the log data to the storage on the primary system and after the log data in the first buffer portion is transmitted to the standby system, transmitting a notification to the standby system, from the primary system, wherein the standby system is configured to process the received log data responsive to receiving the notification from the primary system.

2. The method of claim 1 , further comprising:
determining a connection between the primary system and the standby system has been interrupted, wherein the standby system is configured to, upon reestablishing the connection, delete a portion of the received log data that includes received log data not written to storage at the primary system.

3. The method of claim 1 , wherein the standby system is configured to, upon receiving a request for the standby system to change into a primary role, process an unprocessed portion of the received log data.

4. The method of claim 1 , wherein the notification includes a last-written log position of the primary system, and wherein the last-written log position indicates that the transactional log data was written to storage on the primary system when the last-written log position is at or after a position of the received log data.

5. The method of claim 4 , wherein the standby system is configured to process the received log data by:
identifying at least one portion of the received log data having a position prior to the last-written log position of the received notification; and
processing only the identified at least one portion of the received log data at the standby system.

6. The method of claim 1 , wherein the log data specifies one or more database operations, and wherein the standby system is configured to process the received log data by:
for each database operation of the one or more database operations, executing the database operation against an instance of a database on the standby system.

7. A system, comprising:
a computer processor;
a storage device; and
a memory containing a program that, when executed on the computer processor, performs an operation for data replication, comprising:
allocating, to a first thread on a primary system communicatively coupled to a standby system, a first buffer portion of a transaction log buffer for a first transaction of a plurality of transactions, wherein the transaction log buffer comprises a contiguous area of memory configured to store log data related to the plurality of transactions, wherein the first buffer portion is fully formed when filled with log data related to the first transaction and to be written to storage on the primary system;
allocating, to a second thread on the primary system, a second buffer portion of the transaction log buffer for a second transaction of the plurality of transactions, wherein the second buffer portion is fully formed when filled with log data related to the second transaction and to be written to storage on the primary system;
upon determining that the first buffer portion is fully formed, before the second buffer portion is fully formed, and before the log data in the first buffer portion is written to the storage device, transmitting the log data in the first buffer portion to a standby system;
upon writing the log data to the storage device and after the log data in the first buffer portion is transmitted to the standby system, transmitting a notification to the standby system, from the primary system, wherein the standby system is configured to process the received log data responsive to receiving the notification from the primary system.

8. The system of claim 7 , the operation further comprising:
determining a connection between the primary system and the standby system has been interrupted, wherein the standby system is configured to, upon reestablishing the connection, delete a portion of the received log data that includes received log data not written to storage at the primary system.

9. The system of claim 7 , wherein the standby system is configured to, upon receiving a request for the standby system to change into a primary role, process an unprocessed portion of the received log data.

10. The system of claim 7 , wherein the notification includes a last-written log position of the primary system, and wherein the last-written log position indicates that the transactional log data was written to storage on the primary system when the last-written log position is at or after a position of the received log data.

11. The system of claim 10 , wherein the standby system is configured to process the received log data by: identifying at least one portion of the received log data having a position prior to the last-written log position of the received notification; and
processing only the identified at least one portion of the received log data.

12. The system of claim 7 , wherein the log data specifies one or more database operations, and wherein the standby system is configured to process the received log data by:
for each database operation of the one or more database operations, executing the database operation against an instance of a database on the standby system.

13. A computer program product for data replication, comprising:
a non-transitory computer-readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:
computer readable program code to allocate, to a first thread on a primary system communicatively coupled to a standby system, a first buffer portion of a transaction log buffer for a first transaction of a plurality of transactions, wherein the transaction log buffer comprises a contiguous area of memory configured to store log data related to the plurality of transactions, wherein the first buffer portion is fully formed when filled with log data related to the first transaction and to be written to storage on the primary system;
computer readable program code to allocate, to a second thread on the primary system, a second buffer portion of the transaction log buffer for a second transaction of the plurality of transactions, wherein the second buffer portion is fully formed when filled with log data related to the second transaction and to be written to storage on the primary system;
computer readable program code to, upon determining that the first buffer portion is fully formed, before the second buffer portion is fully formed, and before the log d

Assignees

• Cherkauer Kevin J
• Pearson Steven R
• Xue Xun
• Zheng Roger L Q
• IBM

Inventors

• Cherkauer Kevin J
• Pearson Steven R
• Xue Xun
• Zheng Roger L Q

Classifications

• G06F17/30008

  Physics · mapped topic

• G06F11/2097

  maintaining the standby controller/processing unit updated (initialisation or re-synchronisation thereof G06F11/1658 and subgroups) · CPC title

• G06F17/30368 Primary

  Physics · mapped topic

• G06F17/30377

  Physics · mapped topic

• G06F2201/80

  Database-specific techniques · CPC title