Automatic scaling of computing resources using aggregated metrics

Patent metadata

Field	Value
Publication number	US-2016378552-A1
Application number	US-201514752760-A
Country	US
Kind code	A1
Filing date	Jun 26, 2015
Priority date	Jun 26, 2015
Publication date	Dec 29, 2016
Grant date	—

Abstract

A computing resource monitoring service receives a plurality of measurements for a metric associated with an auto-scale group. Each measurement is associated with metadata for the measurement, which specifies attributes for the measurement. The computing resource monitoring service determines, for each measurement and based at least in part on the metadata, a fully qualified metric identifier for the measurement. The service partitions the plurality of measurements into a plurality of logical partitions associated with one or more in-memory datastores. The service transmits the measurements from the plurality of logical partitions to the one or more datastores for storage of the measurements. These measurements are provided to one or more computing resource managers for the auto-scale group to enable automatic scaling of computing resources of the group based at least in part on the measurements.

First claim

1 . A computer-implemented method, comprising:
under the control of one or more computer systems configured with executable instructions,
receiving, from one or more load balancers for one or more virtual machine instances of an auto-scale group, a plurality of measurements for a metric including metadata specifying attributes of measurements of the plurality of measurements;
utilizing a hash function and the metadata to generate a fully qualified metric identifier for the measurements;
partitioning the measurements based at least in part on the fully qualified metric identifier and a timestamp for the measurements, resulting in a plurality of logical partitions;
transmitting the plurality of measurements from the plurality of logical partitions to one or more aggregator sub-systems for storage within one or more datastores of the one or more aggregator sub-systems;
obtaining, at one or more instance managers for the one or more virtual machine instances, the measurements from the one or more datastores; and
performing automatic scaling of the auto-scale group based at least in part on the obtained measurements by at least changing a number of virtual machine instances in the auto-scale group.

2 . The computer-implemented method of claim 1 , further comprising:
obtaining, from a metric mapping registry, a mapping of the plurality of logical partitions to a plurality of aggregator sub-systems; identifying, for each logical partition of the plurality of logical partitions and based at least in part on the mapping, the corresponding aggregator sub-system; and associating each logical partition with the identified corresponding aggregator sub-system to enable transmission of the measurements of the logical partition to the identified corresponding aggregator sub-system.

3 . The computer-implemented method of claim 1 , further comprising:
receiving, from the one or more instance managers, a request to retrieve one or more measurements, the request including the fully qualified metric identifier; utilizing the fully qualified metric identifier to identify the one or more datastores wherein the one or more measurements are stored; and providing the one or more measurements to the one or more instance managers in response to the request.

4 . The computer-implemented method of claim 1 , further comprising aggregating, within one or more aggregator sub-systems, the measurements with existing measurements within the one or more in-memory datastores to enable storage of the plurality of measurements.

5 . A system, comprising at least one computing device configured to implement one or more services, wherein the one or more services are configured to:
receive a plurality of measurements for a metric associated with a group of computing resources, measurements of the plurality of measurements associated with metadata for the measurements; determine, for the measurements and based at least in part on the metadata, a metric identifier; partition the measurements into a plurality of partitions; transmit the measurements from the plurality of partitions to one or more in-memory datastores for storage; and provide the measurements to one or more computing resource managers for the group to enable modification of one or more computing resources of the group.

6 . The system of claim 5 , wherein the one or more services are configured to:
receive a request to retrieve one or more measurements, the request including the metric identifier and specifying a time period; use the metric identifier and the specified time period to identify the one or more in-memory datastores wherein the one or more measurements are stored; obtain the one or more measurements; and transmit the one or more measurements to the one or more computing resource managers to enable the modification of the one or more computing resources of the group.

7 . The system of claim 5 , wherein the one or more services are further configured to aggregate the measurements with existing measurements of the one or more in-memory datastores for storage of the measurements in the one or more in-memory datastores.

8 . The system of claim 5 , wherein the one or more services are further configured to serialize the measurements in a binary serialization format such that the measurements in the binary serialization format includes the metric identifier, a timestamp for the measurements, a unit of measurement for the measurements, and the measurements themselves.

9 . The system of claim 5 , wherein the one or more services are further configured to, if one or more partitions of the plurality of partitions do not have capacity for the measurements, purge one or more measurements having timestamps earlier than an earliest timestamp for the measurements.

10 . The system of claim 5 , wherein the one or more services are configured to:
obtain a mapping of the plurality of partitions to a plurality of aggregator systems, the plurality of aggregator systems including the one or more in-memory datastores; identify, based at least in part on the mapping, corresponding aggregator systems for partitions of the plurality of partitions; and associate the partitions to the corresponding aggregator systems to enable transmission of the measurements from the partitions to the one or more in-memory datastores.

11 . The system of claim 5 , wherein the one or more services are configured to remove, from the one or more in-memory datastores, expired measurements as a result of an old retention period for the expired measurements being replaced with a newer retention period.

12 . The system of claim 5 , wherein the one or more computing resource managers are configured to provision additional computing resources for the auto-scale group based at least in part on the provided measurements.

13 . A non-transitory computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least:
receive a measurement for a metric associated with a group of computing resources, the measurement associated with metadata for the measurement; determine, based at least in part on the metadata, a metric identifier; select, based at least in part on the metric identifier for the measurement, a partition from a plurality of partitions; transmit the measurement from the selected partition to an in-memory datastore for storage; and provide the measurement to one or more computing resource managers for the group of computing resources to enable one or more operations to be performed on the group of computing resources.

14 . The non-transitory computer-readable storage medium of claim 13 , wherein the one or more computing resource managers are configured to terminate one or more computing resources from the auto-scale group based at least in part on the provided measurement.

15 . The non-transitory computer-readable storage medium of claim 13 , wherein the instructions further cause the computer system to:
determine, based at least in part on a mapping of partitions to aggregator systems, a corresponding aggregator system for the selected partition, the corresponding aggregator system including the in-memory datastore; and associate the partition to the corresponding aggregator system to enable transmission of the measurement from the selected partition to the in-memory datastore.

16 . The non-transitory computer-readable storage medium of claim 13 , wherein the instructions further cause the computer system to:
determine if the selected partitio

Assignees

• Amazon Tech Inc

Inventors

• Taylor Gary
• Dennehy Brian
• Rodriguez Sebastien Alexandre Roland
• Mangas Julio Delgado

Classifications

• G06F9/505 Primary

  considering the load · CPC title

• G06F11/3433

  for load management (allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029) · CPC title

• G06F11/301

  where the computing system is a virtual computing platform, e.g. logically partitioned systems (virtual machines G06F9/45533; logical partitioning of resources G06F9/5077) · CPC title

• G06F9/5077

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F11/3442

  for planning or managing the needed capacity · CPC title