System and method for efficient database record replication using different replication strategies based on the database records

Patent metadata

Field	Value
Publication number	US-9659077-B2
Application number	US-201313920932-A
Country	US
Kind code	B2
Filing date	Jun 18, 2013
Priority date	Jun 18, 2012
Publication date	May 23, 2017
Grant date	May 23, 2017

Abstract

Systems and methods are provided for synchronizing operational data records in a manner that reduces redundant copying of data. A set of operational data records is stored, wherein each data record is assigned a unique ID in ascending order based on a creation time of the data record. A highest unique ID is identified from the operational data records in the set of operational data records. A tolerance number is identified that is indicative of a range of unique IDs that can be processed at a same time such that it cannot be guaranteed that operational data records with unique IDs separated by less than the tolerance number were assigned unique IDs in the order that the operational data records were created. A synchronization ID is calculated comprising subtracting the tolerance number from the highest unique ID. The synchronization ID is transmitted to the source operational data store.

First claim

The invention claimed is:

1. A method for synchronizing operational data records generated during data management operations in a manner that reduces redundant copying of data during synchronization, comprising:
storing, by a host computer, a set of operational data records, wherein each data record is assigned a unique identifier (ID) in ascending order based on a creation time of the data record, the host computer including a processor, the processor coupled to a non-transitory readable medium having instructions executable by the processor; and
performing, by the host computer, a first synchronization operation for the set of operational data records, wherein performing the first synchronization operation comprises:
identifying, by the host computer, a highest unique ID from the operational data records in the set of operational data records associated with a prior synchronization operation, the prior synchronization operation occurring prior to the first synchronization operation;
identifying, by the host computer, a tolerance number that is indicative of a range of unique IDs that can be processed at a same time such that it cannot be guaranteed that operational data records with unique IDs separated by less than the tolerance number were assigned unique IDs in the order that the operational data records were created, such that the operational data records within the tolerance number may not have been transferred during the prior synchronization operation;
calculating, by the host computer, a synchronization ID comprising subtracting the tolerance number from the highest unique ID; and

transmitting, by the host computer, the synchronization ID to a source operational data store to instruct the source operational data store to transmit any operational data records stored at the source operational data store with unique IDs greater than the synchronization ID.

2. The method of claim 1 , further comprising:
identifying a set of unique IDs from the set of operational data records, wherein each unique ID in the set of unique IDs is between the highest unique ID and the synchronization ID; and
transmitting the set of unique IDs to the source operational data store to instruct the source operational data store to not transmit operational data records with the set of unique IDs.

3. The method of claim 1 , further comprising transmitting the highest unique ID to the source operational data store to instruct the source operational data store to not transmit operational data records with the highest unique ID.

4. The method of claim 1 , further comprising calculating the tolerance number based on a number of concurrent transactions allowed and a maximum delay.

5. The method of claim 4 , further comprising doubling the number of concurrent transactions.

6. The method of claim 1 , wherein each unique ID is a monotonically-increasing unique ID.

7. The method of claim 1 , wherein the set of operational data records is a large set of operational data records, comprising over 100,000 operational data records; and the data records in the set of operational data records are not changing once created.

8. A method for synchronizing operational data records generated during data management operations in a manner that reduces redundant copying of data during synchronization, comprising:
storing, by a host computer, a set of operational data records, wherein each data record is assigned a timestamp based on either a creation time, or modification time of the data record, the host computer including a processor, the processor coupled to a non-transitory readable medium having instructions executable by the processor;
identifying, by the host computer, a first backup time associated with a first synchronization operation of the set of operational data records from a source operational data store; and
performing, by the host computer, a subsequent synchronization operation for the set of operational data records, the subsequent synchronization operation associated with a subsequent backup time, the subsequent backup time occurring after the first backup time, wherein performing the subsequent synchronization operation comprises:

calculating, by the host computer, a tolerance number based on an elapsed time that is indicative of a range of timestamps that can be processed at a same time such that it cannot be guaranteed that operational data records with timestamps separated by less than the tolerance number were assigned timestamps in the order that the operational data records were created, modified, or both, such that the operational data records within the tolerance number may not have been transferred at the first backup time;
calculating, by the host computer, a synchronization timestamp comprising subtracting the tolerance number from the first backup time; and
transmitting, by the host computer, the synchronization timestamp to the source operational data store to instruct the source operational data store to transmit any operational data records stored at the source operational data store with timestamps greater than the synchronization timestamp.

9. The method of claim 8 , further comprising receiving a reply comprising:
a set of operational data records, each with a timestamp occurring after the synchronization timestamp; and
a list of unique IDs for each data record stored by the source operational data store.

10. The method of claim 9 , further comprising deleting any operational data records in the set of operational data records with a unique ID that is not in the list of unique IDs.

11. The method of claim 8 , wherein the set of operational data records is a medium set of operational data records, comprising more than 1,000 operational data records but less than 100,000 operational data records.

12. A non-transitory computer readable medium having executable instructions operable to cause an apparatus to:
store a set of operational data records, wherein each data record is assigned a timestamp based on either a creation time, or modification time of the data record;
identify a first backup time associated with a first synchronization operation of the set of operational data records from a source operational data store; and
perform a subsequent synchronization operation for the set of operational data records, the subsequent synchronization operation associated with a subsequent backup time, the subsequent backup time occurring after the first backup time, wherein to perform the subsequent synchronization operation, the apparatus is further caused to:
calculate a tolerance number based on an elapsed time that is indicative of a range of timestamps that can be processed at a same time such that it cannot be guaranteed that operational data records with timestamps separated by less than the tolerance number were assigned timestamps in the order that the operational data records were created, modified, or both, such that the operational data records within the tolerance number may not have been transferred at the first backup time;
calculate a synchronization timestamp comprising subtracting the tolerance number from the first backup time; and
transmit the synchronization timestamp to the source operational data store to instruct the source operational data store to transmit any operational data records stored at the source operational data store with timestamps greater than the synchronization timestamp.

13. The non-transitory computer readable medium of claim 12 , wherein the apparatus is further caused to receive a reply comprising:
a set of operational data records, each with a timestamp occurring after the synchronization timestamp; and
a lis

Assignees

• Actifio Inc

Inventors

• Chen yong-min

Classifications

• G06F3/065

  Replication mechanisms · CPC title

• G06F17/30088

  Physics · mapped topic

• G06F11/1451 Primary

  by selection of backup contents · CPC title

• G06F2201/80

  Database-specific techniques · CPC title

• G06F11/1461

  Backup scheduling policy · CPC title