Providing quality of service (QoS) for replicating datasets

Patent metadata

Field	Value
Publication number	US-12045252-B2
Application number	US-202217573095-A
Country	US
Kind code	B2
Filing date	Jan 11, 2022
Priority date	Sep 13, 2019
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

Providing Quality of Service (QoS) for replicating datasets including: receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository; adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository; selecting, by the target data repository, one or more queues from the plurality of queues; and servicing an operation from each of the selected one or more queues.

First claim

What is claimed is:

1. A method comprising:
receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository, wherein the data stored in the target data repository and the source data repository are distributed across multiple storage nodes;
adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository;
receiving, by the target data repository, a storage operation from a host directed to a volume of the target data repository;
adding, by the target data repository, the storage operation from the host directed to the volume of the target data repository to a second queue, wherein the plurality of queues includes one or more second queues each corresponding to a particular volume of the target data repository, each configured to queue storage operations from a host directed to the particular volume, and each having a higher priority than the first queue for checkpoints of the one or more second queues corresponding to the volume;
selecting, by the target data repository, one or more queues from the plurality of queues that includes one or more first queues for checkpoints and one or more second queues for storage operations from the host having a higher priority than the first queue for checkpoints; and
servicing an operation from each of the selected one or more queues.

2. The method of claim 1 , wherein selecting the one or more queues is based on a respective accumulated cost of serviced operations for each of the plurality of queues.

3. The method of claim 2 , further comprising updating, for each of the selected one or more queues, the respective accumulated cost.

4. The method of claim 3 , wherein the respective accumulated cost is updated based on a priority value based on an entity type.

5. The method of claim 4 , wherein the priority value is further based on a user-defined weight.

6. The method of claim 1 , further comprising:
sending, by the target data repository to the source data repository, data describing a load of the target data repository; and
wherein the data causes the source data repository to adjust one or more parameters for modifying the data set in the source data repository.

7. The method of claim 1 , further comprising modifying, by the source data repository, a data replication mode in response to an age of a last serviced checkpoint exceeding a threshold.

8. A storage system that includes a computer memory and a computer processor, the computer memory including program instructions that, when executed by the computer processor, cause the storage system to carry out the steps of:
receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository, wherein the data stored in the target data repository and the source data repository are distributed across multiple storage nodes;
adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository;
receiving, by the target data repository, a storage operation from a host directed to a volume of the target data repository;
adding, by the target data repository, the storage operation from the host directed to the volume of the target data repository to a second queue, wherein the plurality of queues includes one or more second queues each corresponding to a particular volume of the target data repository, each configured to queue storage operations from a host directed to the particular volume, and each having a higher priority than the first queue for checkpoints of the one or more second queues corresponding to the volume;
selecting, by the target data repository, one or more queues from the plurality of queues that includes one or more first queues for checkpoints and one or more second queues for storage operations from the host having a higher priority than the first queue for checkpoints; and
servicing an operation from each of the selected one or more queues.

9. The storage system of claim 8 , wherein selecting the one or more queues is based on a respective accumulated cost of serviced operations for each of the plurality of queues.

10. The storage system of claim 9 , wherein the program instructions, when executed by the computer processor, further cause the storage system to carry out the step of updating, for each of the selected one or more queues, the respective accumulated cost.

11. The storage system of claim 10 , wherein the respective accumulated cost is updated based on a priority value based on an entity type.

12. The storage system of claim 11 , wherein the priority value is further based on a user-defined weight.

13. The storage system of claim 8 , wherein the program instructions, when executed by the computer processor, further cause the storage system to carry out the steps of:
sending, by the target data repository to the source data repository, data describing a load of the target data repository; and
wherein the data causes the source data repository to adjust one or more parameters for modifying the data set in the source data repository.

14. The storage system of claim 8 , wherein the program instructions, when executed by the computer processor, further cause the storage system to carry out the steps of modifying, by the source data repository, a data replication mode in response to an age of a last serviced checkpoint exceeding a threshold.

15. A computer program product disposed on a non-transitory computer readable medium, the computer program product including computer program instructions that, when executed, carry out the steps of:
receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository, wherein the data stored in the target data repository and the source data repository are distributed across multiple storage nodes;
adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository;
receiving, by the target data repository, a storage operation from a host directed to a volume of the target data repository;
adding, by the target data repository, the storage operation from the host directed to the volume of the target data repository to a second queue, wherein the plurality of queues includes one or more second queues each corresponding to a particular volume of the target data repository, each configured to queue storage operations from a host directed to the particular volume, and each having a higher priority than the first queue for checkpoints of the one or more second queues corresponding to the volume;
selecting, by the target data repository, one or more queues from the plurality of queues that includes one or more first queues for checkpoints and one or more second queues for storage operations from the host having a higher priority than the first queue for checkpoints; and
servicing an operation from each of the selected one or more queues.

Assignees

• Pure Storage Inc

Inventors

• Sonner Daniel
• He Jun
• Wang Zong
• Colgrove John
• Fay Matthew

Classifications

• G06F2201/88

  Monitoring involving counting · CPC title

• G06F11/2097

  maintaining the standby controller/processing unit updated (initialisation or re-synchronisation thereof G06F11/1658 and subgroups) · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F11/3034

  where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• G06F16/275

  Synchronous replication · CPC title