Automatic scaling of computing resources using aggregated metrics

Patent metadata

Field	Value
Publication number	US-9880880-B2
Application number	US-201514752760-A
Country	US
Kind code	B2
Filing date	Jun 26, 2015
Priority date	Jun 26, 2015
Publication date	Jan 30, 2018
Grant date	Jan 30, 2018

Abstract

A computing resource monitoring service receives a plurality of measurements for a metric associated with an auto-scale group. Each measurement is associated with metadata for the measurement, which specifies attributes for the measurement. The computing resource monitoring service determines, for each measurement and based at least in part on the metadata, a fully qualified metric identifier for the measurement. The service partitions the plurality of measurements into a plurality of logical partitions associated with one or more in-memory datastores. The service transmits the measurements from the plurality of logical partitions to the one or more datastores for storage of the measurements. These measurements are provided to one or more computing resource managers for the auto-scale group to enable automatic scaling of computing resources of the group based at least in part on the measurements.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, from one or more load balancers for one or more virtual machine instances of an auto-scale group, a plurality of measurements for a metric including metadata specifying attributes of measurements of the plurality of measurements;
utilizing a hash function and the metadata to generate a fully qualified metric identifier for the measurements;
partitioning the measurements based at least in part on the fully qualified metric identifier and a timestamp for the measurements, resulting in a plurality of logical partitions, wherein at least a portion of the fully qualified metric identifier identifies a logical partition of the plurality of logical partitions, while another portion of the fully qualified metric identifier is used for unique identification of the measurements within the logical partition;
transmitting the plurality of measurements from the plurality of logical partitions to one or more aggregator sub-systems for storage within one or more datastores of the one or more aggregator sub-systems;
obtaining, at one or more instance managers for the one or more virtual machine instances, the measurements from the one or more datastores; and
performing automatic scaling of the auto-scale group based at least in part on the obtained measurements by at least changing a number of virtual machine instances in the auto-scale group.

2. The computer-implemented method of claim 1 , further comprising:
obtaining, from a metric mapping registry, a mapping of the plurality of logical partitions to a plurality of aggregator sub-systems;
identifying, for each logical partition of the plurality of logical partitions and based at least in part on the mapping, the corresponding aggregator sub-system; and
associating each logical partition with the identified corresponding aggregator sub-system to enable transmission of the measurements of the logical partition to the identified corresponding aggregator sub-system.

3. The computer-implemented method of claim 1 , further comprising:
receiving, from the one or more instance managers, a request to retrieve one or more measurements, the request including the fully qualified metric identifier;
utilizing the fully qualified metric identifier to identify the one or more datastores wherein the one or more measurements are stored; and
providing the one or more measurements to the one or more instance managers in response to the request.

4. The computer-implemented method of claim 1 , further comprising aggregating, within one or more aggregator sub-systems, the measurements with existing measurements within the one or more in-memory datastores to enable storage of the plurality of measurements.

5. A system, comprising at least one computing device configured to implement one or more services, wherein the one or more services are configured to:
receive a plurality of measurements for a metric associated with a group of computing resources, measurements of the plurality of measurements associated with metadata for the measurements;
determine, using a hash function and the metadata, a metric identifier for the measurements;
partition the measurements into a plurality of partitions based at least in part on the metric identifier and a timestamp of the measurements, wherein at least a portion of the metric identifier identifies a partition of the plurality of partitions, while another portion of the metric identifier is used for unique identification of the measurements within partition;
transmit the measurements from the plurality of partitions to one or more in-memory datastores for storage; and
provide the measurements to one or more computing resource managers for the group to enable modification of one or more computing resources of the group.

6. The system of claim 5 , wherein the one or more services are configured to:
receive a request to retrieve one or more measurements, the request including the metric identifier and specifying a time period;
use the metric identifier and the specified time period to identify the one or more in-memory datastores wherein the one or more measurements are stored;
obtain the one or more measurements; and
transmit the one or more measurements to the one or more computing resource managers to enable the modification of the one or more computing resources of the group.

7. The system of claim 5 , wherein the one or more services are further configured to aggregate the measurements with existing measurements of the one or more in-memory datastores for storage of the measurements in the one or more in-memory datastores.

8. The system of claim 5 , wherein the one or more services are further configured to serialize the measurements in a binary serialization format such that the measurements in the binary serialization format includes the metric identifier, a timestamp for the measurements, a unit of measurement for the measurements, and the measurements themselves.

9. The system of claim 5 , wherein the one or more services are further configured to, if one or more partitions of the plurality of partitions do not have capacity for the measurements, purge one or more measurements having timestamps earlier than an earliest timestamp for the measurements.

10. The system of claim 5 , wherein the one or more services are configured to:
obtain a mapping of the plurality of partitions to a plurality of aggregator systems, the plurality of aggregator systems including the one or more in-memory datastores;
identify, based at least in part on the mapping, corresponding aggregator systems for partitions of the plurality of partitions; and
associate the partitions to the corresponding aggregator systems to enable transmission of the measurements from the partitions to the one or more in-memory datastores.

11. The system of claim 5 , wherein the one or more services are configured to remove, from the one or more in-memory datastores, expired measurements as a result of an old retention period for the expired measurements being replaced with a newer retention period.

12. The system of claim 5 , wherein the one or more computing resource managers are configured to provision additional computing resources for the auto-scale group based at least in part on the provided measurements.

13. A non-transitory computer-readable storage medium having stored thereon executable instructions that, as a result of execution by one or more processors of a computer system, cause the computer system to at least:
receive a measurement for a metric associated with a group of computing resources, the measurement associated with metadata for the measurement;
determine, usinq a hash function and the metadata, a metric identifier for the measurement;
select, based at least in part on the metric identifier for the measurement and a timestamp for the measurement, a partition from a plurality of partitions, wherein at least a portion of the metric identifier identifies the selected partition, while another portion of the metric identifier is used for unique identification of the measurement within the partition;
transmit the measurement from the selected partition to an in-memory datastore for storage; and
provide the measurement to one or more computing resource managers for the group of computing resources to enable one or more operations to be performed on the group of computing resources.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the one or more computing resource managers are configured to terminate one or more computing resources from the auto-scale group based at least in pa

Assignees

• Amazon Tech Inc

Inventors

• Taylor Gary
• Dennehy Brian
• Rodriguez Sebastien Alexandre Roland
• Mangas Julio Delgado

Classifications

• G06F11/3082

  the data filtering being achieved by aggregating or compressing the monitored data · CPC title

• G06F9/505 Primary

  considering the load · CPC title

• G06F2201/815

  Virtual · CPC title

• G06F11/301

  where the computing system is a virtual computing platform, e.g. logically partitioned systems (virtual machines G06F9/45533; logical partitioning of resources G06F9/5077) · CPC title

• G06F9/5077

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title