Utilizing mutiple snapshot sources for creating new copy of volume in a networked environment wherein additional snapshot sources are reserved with lower performance levels than a primary snapshot source

Patent metadata

Field	Value
Publication number	US-11068192-B1
Application number	US-201916365442-A
Country	US
Kind code	B1
Filing date	Mar 26, 2019
Priority date	Mar 26, 2019
Publication date	Jul 20, 2021
Grant date	Jul 20, 2021

Abstract

Aspects relate to techniques for using read-only volume replicas in a distributed computing environment to enable over-subscription on server performance. In order to provide a good customer experience, the I/O handling replicas of a volume are typically reserved at a high percentage of the customer's desired performance. A read-only replica of the volume does not serve user I/O, and can therefore be reserved at a much lower percentage of desired performance. Particularly, as the number of read-only replicas increases, the performance reservation can be lowered due to the statistical likelihood that the server(s) hosting at least one read-only replica will have sufficient performance to support the desired reads during new volume creation (even though that performance has not been fully reserved). Aspects relate to techniques for selecting among read-only replicas to serve reads during creation of a new volume copy.

First claim

What is claimed is:

1. A system comprising:
a control plane server;
a first set of servers in communication with the control plane server and having collectively stored thereon a first number of partitions of a primary replica of a volume and the first number of partitions of a secondary replica, wherein the volume has a plurality of additional replicas, and wherein the first set of servers is configured with computer-executable instructions to at least:
process user-initiated read or write operations on the volume,
reserve a first level of performance for the primary replica to use in processing the user-initiated read or write operations on the volume, wherein the first level of performance is one or more of Input/Output Operations Per Second (IOPS), network bandwidth, processing, or memory, and
reserve the first level of reserved performance for the secondary replica;
a second set of servers in communication with the control plane server and having collectively stored thereon a second number of partitions of a first additional replica of the plurality of additional replicas, wherein the second number of partitions is greater than the first number of partitions, and wherein the second set of servers is configured with computer-executable instructions to at least:
reserve, based at least in part on a quantity of the plurality of additional replicas, a second level of performance, for the first additional replica to use, that is less than the first level of performance, and
in response to a first request for a partition from the control plane server, transmit a partition of the first additional replica to the control plane server; and
a third set of servers in communication with the control plane server and having collectively stored thereon the second number of partitions of a second additional replica of the plurality of additional replicas, wherein the third set of servers is configured with computer-executable instructions to at least:
reserve the second level of performance for the second additional replica to use, and
in response to a second request from the control plane server, transmit a partition of the second additional replica to the control plane server, wherein the partition of the second additional replica and the partition of the first additional replica correspond to different partitions of the volume;
wherein the control plane server is configured to at least:
detect that the primary replica has failed,
based at least in part on detecting that the primary replica has failed,
transmit the first request to the second set of servers, and
transmit the second request to the third set of servers,
receive the partition of the first additional replica from the second set of servers,
receive the partition of the second additional replica from the third set of servers, and
combine at least the partition of the first additional replica and the partition of the second additional replica to generate a new copy of the volume for processing the user-initiated read or write operations on the volume.

2. The system of claim 1 , further comprising a fourth set of servers, and wherein the control plane server is further configured to store the new copy of the volume on the fourth set of servers.

3. The system of claim 2 , wherein the control plane server is further configured to at least:
identify a first level of current usage of performance of the second set of servers;
identify a second level of current usage of performance of the third set of servers; and
select the first and second additional replicas for use in creating the new copy of the volume based on the first level of current usage of performance of the second set of servers and the second level of current usage of performance of the third set of servers.

4. The system of claim 1 , wherein the control plane server is further configured to at least:
determine the quantity of the plurality of additional replicas; and
determine the second level of performance, wherein the second level of performance is inversely related to the quantity of the plurality of additional replicas.

5. A computer-implemented method for efficient management of a storage volume in a distributed computing environment, the computer-implemented method comprising:
determining a first level of performance for a primary replica of a volume of data, wherein the primary replica is stored as a first number of partitions distributed among a first set of servers;
receiving an instruction to generate an additional replica of the volume for processing reads and writes to the volume with the first level of reserved performance;
creating a first secondary replica of the volume, wherein the first secondary replica is stored as a second number of partitions distributed among a second set of servers, and wherein the second number of partitions is greater than the first number of partitions;
creating a second secondary replica of the volume, wherein the second secondary replica is stored as the second number of partitions distributed among a third set of servers;
determining a second level of performance for each of the first secondary replica of the volume and the second secondary replica of the volume, wherein the second level of performance is less than the first level of performance; and
in response to receiving an instruction to create a new copy of the volume,
obtaining a partition of the first secondary replica,
obtaining a partition of the second secondary replica, wherein the partition of the second secondary replica and the partition of the first secondary replica correspond to different partitions of the volume, and
combining at least the partition of the first secondary replica and the partition of the second secondary replica to generate the new copy of the volume.

6. The computer-implemented method of claim 5 , further comprising:
selecting, based on current usage of performance of the second set of servers and the third set of servers, the first secondary replica and the second secondary replica.

7. The computer-implemented method of claim 6 , further comprising, for the second set of servers and the third set of servers, determining the current usage of performance based at least on Input/Output Operations Per Second (IOPS) of a plurality of volumes stored on the second set of servers and the third set of servers, wherein the second level of performance specifies a reserved level of IOPS.

8. The computer-implemented method of claim 5 , further comprising:
creating the new copy of the volume on a fourth set of servers.

9. The computer-implemented method of claim 8 , further comprising:
selecting the first secondary replica based on determining that usage of performance of the second set of servers is less than usage of performance of a fifth set of servers that store a third secondary replica; and
selecting the second secondary replica based on determining that usage of performance of the third set of server is less than usage of performance of the fifth set of servers.

10. The computer-implemented method of claim 8 , further comprising:
selecting the first secondary replica at least partly based on determining a proximity between the second set of servers and the fourth set of servers; and
selecting the second secondary replica at least partly based on determining a proximity between the third set of servers and the fourth set of servers.

11. The computer-implemented method of claim 5 , wherein the first level of performance is one or more of Input/Output Operations Per Second (IOPS), network bandwidth, processing, or memory of the first set of servers, the computer-implemented method further comprising instructing r

Assignees

• Amazon Tech Inc

Inventors

• Greenwood Christopher Magee
• Thompson James Michael
• Kostic Igor A

Classifications

• G06F3/065 Primary

  Replication mechanisms · CPC title

• G06F3/0613

  in relation to throughput · CPC title

• G06F3/0617

  in relation to availability · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title