Automatic scaling of resource instance groups within compute clusters

Patent metadata

Field	Value
Publication number	US-2016323377-A1
Application number	US-201514702080-A
Country	US
Kind code	A1
Filing date	May 1, 2015
Priority date	May 1, 2015
Publication date	Nov 3, 2016
Grant date	—

Abstract

A service provider may apply customer-selected or customer-defined auto-scaling policies to a cluster of resources (e.g., virtualized computing resource instances or storage resource instances in a MapReduce cluster). Different policies may be applied to different subsets of cluster resources (e.g., different instance groups containing nodes of different types or having different roles). Each policy may define an expression to be evaluated during execution of a distributed application, a scaling action to take if the expression evaluates true, and an amount by which capacity should be increased or decreased. The expression may be dependent on metrics emitted by the application, cluster, or resource instances by default, metrics defined by the client and emitted by the application, or metrics created through aggregation. Metric collection, aggregation and rules evaluation may be performed by a separate service or by cluster components. An API may support auto-scaling policy definition.

First claim

What is claimed is:

1 . A distributed computing system, comprising:
a plurality of compute nodes, each compute node comprising at least one processor and a memory; and an interface; wherein the distributed computing system implements a distributed computing service; wherein the plurality of compute nodes are configured as a cluster of compute nodes according to a MapReduce distributed computing framework, wherein the cluster is configured to execute a distributed application; wherein the distributed computing service is configured to:
receive, through the interface from a client of the distributed computing service, input defining an expression that, when evaluated true, represents a trigger condition for performing an automatic scaling operation on the cluster and input specifying a scaling action to be taken in response to the expression evaluating true, wherein the expression is dependent on values of one or more metrics generated during execution of the distributed application;
collect, during execution of the distributed application, the one or more metrics;
determine, during execution of the distributed application and dependent on the collected metrics, that the expression evaluates true; and
initiate, in response to the determination, performance of the automatic scaling operation on the cluster, wherein the automatic scaling operation comprises an operation to add one or more compute nodes to the cluster or an operation to remove one or more compute nodes from the cluster.

2 . The system of claim 1 ,
wherein the plurality of compute nodes comprises two or more groups of compute nodes, each of which includes a non-overlapping subset of the plurality of compute nodes; wherein the inputs received through the interface define an automatic scaling policy; wherein the inputs received through the interface further comprise input identifying one or more of the two or more groups of compute nodes as groups of compute nodes to which the automatic scaling policy applies; and wherein to initiate performance of the automatic scaling operation on the cluster, the distributed computing service is configured to initiate performance of an operation to add one or more compute nodes to one of the identified groups of compute nodes or an operation to remove one or more compute nodes from one of the identified groups of compute nodes.

3 . The system of claim 1 ,
wherein the plurality of compute nodes comprises two or more groups of compute nodes, each of which includes a non-overlapping subset of the plurality of compute nodes; wherein the inputs received through the interface define an automatic scaling policy; and wherein the automatic scaling policy specifies that the scaling action to be taken in response to the expression evaluating true comprises an operation to add a new group of compute nodes to the plurality of compute nodes or an operation to remove one of the two or more groups of compute nodes from the plurality of compute nodes.

4 . The system of claim 1 ,
wherein the distributed application is configured to emit one or more application-specific metrics that were defined by the client of the distributed computing service; and wherein the expression is dependent on at least one of the one or more application-specific metrics.

5 . The system of claim 1 ,
wherein the expression is dependent on one or more metrics that are emitted by the cluster or by one or more of the compute nodes by default while operating in the distributed computing system.

6 . The system of claim 1 ,
wherein to collect, during execution of the distributed application, the one or more metrics, the distributed computing service is configured to:
receive one or more metrics from a respective monitoring component on each of two or more of the plurality of compute nodes; and
aggregate the metrics received from the respective monitoring components to generate an aggregate metric for the two or more compute nodes; and
wherein the expression is dependent on the aggregate metric.

7 . A method, comprising:
performing, by one or more computers:
creating a cluster of computing resource instances, wherein the cluster comprises two or more instance groups, each comprising one or more computing resource instances;
receiving input associating an automatic scaling policy with one of the two or more instance groups, wherein the automatic scaling policy defines a condition that, when met, triggers the performance of an automatic scaling operation on the one of the two or more instance groups that changes the number of computing resource instances in the one of the two or more instance groups;
detecting, during execution of a distributed application on the cluster, that the trigger condition has been met; and
initiating, in response to said detecting, performance of the automatic scaling operation on the one of the two or more instance groups.

8 . The method of claim 7 ,
wherein the trigger condition comprises an expression that, when evaluated true, triggers the performance of the automatic scaling operation on the one of the two or more instance groups, and wherein the expression is dependent on one or more metrics generated during execution of the distributed application on the cluster.

9 . The method of claim 7 ,
wherein the trigger condition comprises an expression that, when evaluated true, triggers the performance of the automatic scaling operation on the one of the two or more instance groups, and wherein the expression is dependent on a day of the week, a date, a time of day, an elapsed period of time, or an estimated period of time.

10 . The method of claim 7 , further comprising:
receiving input associating another automatic scaling policy with another one of the two or more instance groups, wherein the other automatic scaling policy defines a second condition that, when met, triggers the performance of a second automatic scaling operation on the other one of the two or more instance groups that changes the number of computing resource instances in the other one of the two or more instance groups; detecting, during execution of the distributed application on the cluster, that the second trigger condition has been met; and in response to detecting that the second trigger condition has been met, initiating performance of the second automatic scaling operation on the other one of the two or more instance groups.

11 . The method of claim 7 ,
wherein the automatic scaling operation comprises an operation to add capacity to the one of the two or more instance groups.

12 . The method of claim 7 ,
wherein the automatic scaling operation comprises an operation to remove capacity from the one of the two or more instance groups.

13 . The method of claim 12 ,
wherein the method further comprises:
determining which of the one or more of the computing resource instances to remove from the one of the two or more instance groups; and
removing the determined one or more of the computing resource instances from the one of the two or more instance groups; and
wherein said determining is dependent on one or more of: determining that one of the computing resource instances in the one of the two or more instance groups stores data that would be lost if the computing resource were removed, determining that removal of one of the computing resource instances in the one of the two or more instance groups would result in a replication requirement or quorum requirement not being met, determining that one of the computing resource nodes in the one of the two or more instance groups has been decommissioned, determining that one of the comp

Assignees

• Amazon Tech Inc

Inventors

• Einkauf Jonathan Daly
• Natali Luca
• Kalathuru Bhargava Ram
• Baji Saurabh Dileep
• Sinha Abhishek Rajnikant

Classifications

• G06F9/5083

  Techniques for rebalancing the load in a distributed system · CPC title

• H04L67/10

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title

• H04L41/0893

  Assignment of logical groups to network elements · CPC title

• H04L43/0876

  Network utilisation, e.g. volume of load or congestion level · CPC title

• H04L41/0895

  Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements · CPC title