Optimizing geometry based on workload characteristics

Patent metadata

Field	Value
Publication number	US-10474372-B1
Application number	US-201414536458-A
Country	US
Kind code	B1
Filing date	Nov 7, 2014
Priority date	Nov 7, 2014
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

A system and method for provisioning a volume and repartitioning a provisioned volume based at least in part on a workload. A request to provision a volume of a specified size is received, a first set of partition options is determined based at least in part on the specified size, and second set of partition options is determined based at least in part on one or more performance characteristics. A volume partitioning is determined based at least in part on an intersection of a number of partitions between the first set of partition options and the second set of partition options, and further based at least in part on a set of optimization criteria. Based at least in part on tracked usage of the volume or a repartition request, a determination is made to repartition the volume such that the partitioning scheme fulfills a set of optimization criteria.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
tracking input/output operations of a first volume provided by a data storage service to a customer of a computing resource service provider, wherein the first volume is a logical storage space, comprised of a set of data storage partitions within a data storage system according to a first partitioning scheme, in which data objects may be stored and wherein the first volume is attached to a computer system instance of a computer system service such that an operating system of the computer system instance recognizes the first volume and is able to perform the input/output operations with the first volume;
analyzing the tracked input/output operations to determine an input/output profile for the first volume;
determining, based at least in part on the determined input/output profile, to change the first partitioning scheme;
determining a second partitioning scheme based at least in part on the determined input/output profile to fulfill a set of optimization criteria that at least specifies a size and number of partitions for a set of persistent computer-readable storage media, wherein the set of optimization criteria reduces an amount of resources of the data storage service consumed while meeting or exceeding a service-level agreement threshold;
provisioning a second volume comprised of the size and number of partitions specified by the second partitioning scheme;
copying the data objects from the first volume and the first partitioning scheme to the second volume and the second partitioning scheme;
detaching the first volume from the computer system instance; and
attaching the second volume to the computer system instance such that the operating system of the computer system instance recognizes the second volume and is able to perform the input/output operations with the second volume and the second partitioning scheme instead of the first volume and the first partitioning scheme.

2. The computer-implemented method of claim 1 , wherein the set of optimization criteria includes criteria to distribute frequently accessed data blocks of a block-level storage device among multiple partitions.

3. The computer-implemented method of claim 1 , wherein the set of optimization criteria include criteria for meeting one or more performance criteria.

4. The computer-implemented method of claim 1 , wherein determining the second partitioning scheme comprises:
determining a first set of partition options based at least in part on volume size criteria;
determining a second set of partition options based at least in part on desired performance characteristics; and
determining the second partitioning scheme based at least in part on the first set of partition options and the second set of partition options.

5. The computer-implemented method of claim 4 , wherein the desired performance characteristics are one of desired input/output operations per second, desired throughput, or desired latency.

6. A system, comprising:
one or more processors; and
memory including instructions that, as a result of execution by the one or more processors, cause the system to:
receive a request to provision a volume of a specified size;
determine a first set of partition options based at least in part on the specified size;
determine a second set of partition options based at least in part on one or more performance characteristics;
determine a partition scheme based at least in part on an intersection of a number of partitions between the first set of partition options and the second set of partition options and further based at least in part on a set of optimization criteria, a type of volume being provisioned, and input/output profiles of other volumes of that type; and
provision the volume according to the determined partition scheme.

7. The system of claim 6 , wherein the instructions that cause the system to receive a request, further includes instructions that, as a result of execution by the one or more processors, cause the system to receive a request to provision the volume with the one or more performance characteristics.

8. The system of claim 6 , wherein the set of optimization criteria includes criteria for keeping wasted space at or below a threshold.

9. The system of claim 6 , wherein the set of optimization criteria include one or more criteria to spread utilization uniformly among partitions of the volume or reduce a likelihood of correlated load.

10. The system of claim 6 , wherein the one or more performance characteristics are determined based at least in part on the specified size.

11. The system of claim 6 , wherein the volume is provided by a data storage service to a customer of a computing resource service provider and the partition scheme is further determined based at least in part on input/output profiles of other volumes that have been provided to the customer by the data storage service.

12. A non-transitory computer-readable storage medium having stored thereon executable instructions that, as a result of execution by one or more processors of a computer system, cause the computer system to at least:
determine an input/output profile for a first volume based at least in part on tracked input/output operations of the first volume, wherein the first volume is comprised of a set of data storage partitions according to a first partitioning scheme;
determine a second partitioning scheme for the first volume such that the second partitioning scheme fulfills a set of optimization criteria to reduce an amount of data storage resources consumed, wherein the second partitioning scheme at least specifies a size and number of partitions;
provision a second volume comprised of the size and number of partitions specified by the second partitioning scheme; and
replace the first volume and the first partitioning scheme with the second volume and the second partitioning scheme.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions further include instructions that, as a result of execution by one or more processors of a computer system, cause the computer system to:
attach the second volume in parallel with the first volume such that the first volume and the second volume appear as a single volume;
render the first volume as read-only; and
copy data from the first volume to the second volume while allowing read operations to be performed on the first volume and read-write operations to be performed on the second volume.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions that determine to repartition the first volume include instructions that, as a result of execution by the one or more processors, cause the computer system to determine to repartition the first volume based at least in part on receiving a request to change one or more of a size of the first volume or a number of input/output operations per second supported by the first volume.

15. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions that replace the first volume with the second volume include instructions that, as a result of execution by the one or more processors, cause the computer system to:
detach the first volume from the computer system such that the first volume is unrecognized by the computer system and the computer system is unable to perform input/output operations with the first volume; and
attach the second volume to the computer system such that the computer system recognizes the second volume and is able to perform the input/output operations with

Assignees

• Amazon Tech Inc

Inventors

• Olson Marc Stephen
• Brooker Marc John
• Goyal Tarun
• Tripathi Arpit

Classifications

• G06F3/0604 Primary

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/0671

  In-line storage system · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0617 Primary

  in relation to availability · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title