Method, apparatus and computer storage medium for data synchronization

Patent metadata

Field	Value
Publication number	US-11593233-B2
Application number	US-201916576959-A
Country	US
Kind code	B2
Filing date	Sep 20, 2019
Priority date	Oct 31, 2018
Publication date	Feb 28, 2023
Grant date	Feb 28, 2023

Abstract

Techniques provide for data synchronization. For example, such a technique may involve: obtaining respective synchronization characteristics of a group of synchronization jobs to be processed, each synchronization characteristic indicating at least one of an expected completion time instant and an amount of data to be synchronized of a corresponding synchronization job; prioritizing the group of the synchronization jobs based on the synchronization characteristics; and controlling execution of the group of the synchronization jobs based on a result of the prioritizing. Accordingly, high priority is given to the synchronization jobs which can be rapidly completed thereby improving the Recovery Point Objective (RPO) achievement rate before occurrence of a failure.

First claim

We claim:

1. A method for data synchronization within a storage system having a scheduler, a first storage device, and a second storage device, comprising:
obtaining sets of respective synchronization characteristics of a plurality of synchronization jobs of a group of synchronization jobs to be executed, each set of synchronization characteristics indicating an expected-completion time instant and an amount of data to be synchronized of a corresponding synchronization job in a plurality of synchronization sessions;
prioritizing the group of the synchronization jobs to provide a ranking, the ranking indicating an order in which to execute the synchronization jobs of the group of synchronization jobs; and
executing the group of the synchronization jobs in the plurality of synchronization sessions according to the ranking beginning with a highest priority synchronization job and ending with a lowest priority synchronization job, the group of synchronization jobs synchronizing data between the first storage device and the second storage device during the plurality of synchronization sessions;
wherein prioritizing the group of the synchronization jobs includes:
normalizing lengths of available time for executing the synchronization jobs and amounts of data to be synchronized of the synchronization jobs, each of the lengths of available time for executing the synchronization jobs indicating an amount of time until a respective expected completion time instant from a current time instant of the storage system; and
determining the ranking based on weighted sums of the normalized lengths of available times and the normalized amounts of data to be synchronized; and

wherein the scheduler is constructed and arranged to
direct new synchronization jobs at the storage system to an execution path of the storage system while the number of current active synchronization jobs in the synchronization group is below a predetermined threshold,
queue the new synchronization jobs within a waiting queue of the storage system while the number of current active synchronization jobs in the synchronization group is not below the predetermined threshold, and
reorder the synchronization jobs residing within the waiting queue.

2. The method according to claim 1 , wherein obtaining the synchronization characteristics comprises:
obtaining respective Recovery Point Objectives RPOs of the group of synchronization jobs and the current time instant of the storage system; and
determining the expected completion time instant of the corresponding synchronization job based on the RPO of the corresponding synchronization job and the current time instant of the storage system.

3. The method according to claim 1 , further comprising:
in response to arrival of each new synchronization job at the storage system, determining the number of current active synchronization jobs in the group of synchronization jobs; and
in response to the number reaching a threshold, adding the new synchronization job into the group of synchronization jobs.

4. The method according to claim 1 , wherein executing the group of the synchronization jobs includes:
in response to the number of the current active synchronization jobs being below the predetermined threshold, selecting from the group of synchronization jobs a synchronization job with the highest priority as a synchronization job to be executed based on the result of the prioritizing; and
executing the selected synchronization job.

5. The method according to claim 1 , wherein each weighted sum of the set of weighted sums is based on a length of available time for executing and an amount of data to be synchronized for a respective synchronization job of the group of synchronization jobs.

6. The method according to claim 1 , wherein the storage system is tasked with performing data replication between the first storage device and the second storage device in accordance with a predefined Recovery Point Objective (RPO) requirement; and
wherein the group of synchronization jobs to be executed are synchronization jobs awaiting execution to replicate data from the first storage device to the second storage device during the plurality of synchronization sessions.

7. The method according to claim 1 , wherein obtaining the respective synchronization characteristics includes calculating, as an expected completion time instant of a particular synchronization job, a latest time before an expiry of a Recovery Point Objective (RPO) associated with the particular synchronization job; and
wherein the method further comprises calculating, as a length of available time for executing the particular synchronization job and after obtaining the respective synchronization characteristics, a difference between the expected completion time instant of the particular synchronization job and the current time instant.

8. The method according to claim 1 , further comprising:
after prioritizing the group of the synchronization jobs, updating the group of the synchronization jobs to include another synchronization job; and
in response to updating the group of synchronization jobs, updating the lengths of available times for executing the synchronization jobs and reprioritizing the group of the synchronization jobs based on the updated lengths of available times.

9. An electronic device, comprising:
at least one processor;
a scheduler; and
at least one memory coupled to the at least one processor and having computer programs stored therein, the computer programs, when executed by the at least one processor, causing the electronic device to:
obtain sets of respective synchronization characteristics of a plurality of synchronization jobs of a group of synchronization jobs to be executed, each set of synchronization characteristics indicating an expected completion time instant and an amount of data to be synchronized of a corresponding synchronization job in a plurality of synchronization sessions;
prioritize the group of the synchronization jobs to provide a ranking, the ranking indicating an order in which to execute the synchronization jobs of the group of synchronization jobs; and
execute the group of the synchronization jobs in the plurality of synchronization sessions according to the ranking beginning with a highest priority synchronization job and ending with a lowest priority synchronization job, the group of synchronization jobs synchronizing data between a first storage device and a second storage device during the plurality of synchronization sessions;

wherein the electronic device, when prioritizing the group of the synchronization jobs, is constructed and arranged to:
normalize lengths of available time for executing the synchronization jobs and amounts of data to be synchronized of the synchronization jobs, each of the lengths of available time for executing the synchronization jobs indicating an amount of time until a respective expected completion time instant from a current time instant of the electronic device; and
determine the ranking based on weighted sums of the normalized lengths of available times and the normalized amounts of data to be synchronized; and

wherein the scheduler is constructed and arranged to
direct new synchronization jobs at the electronic device to an execution path of the electronic device while the number of current active synchronization jobs in the synchronization group is below a predetermined threshold,
queue the new synchronization jobs within a waiting queue of the electronic device while the number of current active synchronization jobs in the synchronization group is not below the predetermined threshold, and
reorder the synchronization jobs residing within the waiting queue.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Du Fang
• Xiao Pan
• Chen Xu
• Chen Peilei

Classifications

• G06F11/1658 Primary

  Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F9/4837 Primary

  time dependent · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F9/5038

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title