Predictive load balancer resource management

Patent metadata

Field	Value
Publication number	US-10409649-B1
Application number	US-201414502424-A
Country	US
Kind code	B1
Filing date	Sep 30, 2014
Priority date	Sep 30, 2014
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

Computing resource service providers allow customers to execute computer systems on hardware provided by the computing resource service provider. The hardware may be distributed between various geographic locations connected by a network. A load balancer may be provided to distribute traffic between the computer systems. Furthermore, computing resource service provider may cause computing resources to be allocated or deallocated to the load balancer based at least in part on various attributes of the computer systems the load balancer is responsible for distributing traffic to. The various attributes may include a capacity of the computer systems.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
under the control of one or more computer systems configured with executable instructions,
assigning a set of backend computer systems to a load balancer such that the load balancer is configured to distribute requests to the set of backend computer systems;
generating a predictive model for predicting a capacity need of the load balancer, by at least:
tracking an aggregate capacity of the set of backend computer systems;
determining, based at least in part on the tracked aggregate capacity of the set of backend computer systems, a ratio of the capacity need of the load balancer to the tracked aggregate capacity of the set of backend computer systems;

monitoring the aggregate capacity to determine a current backend capacity; and
using the predictive model to modify a load balancer capacity in accordance with the predictive model in response to the current backend capacity exceeding a threshold.

2. The computer-implemented method of claim 1 , wherein using the predictive model further comprises using a matching table indicating the ratio of the aggregate capacity of the set of backend computer systems to the capacity need of the load balancer based at least in part on a utilization rate of the set of backend computer systems to modify the load balancer capacity based at least in part on information indicated in the matching table.

3. The computer-implemented method of claim 1 , wherein tracking the aggregate capacity of the set of backend computer systems further includes tracking a number of requests directed to the set of backend computer systems.

4. The computer-implemented method of claim 1 , wherein modifying the load balancer capacity in accordance with the predictive model further includes adjusting the load balancer capacity to a percentage of a maximum tracked aggregate capacity of the set of backend computer systems.

5. A system, comprising:
one or more processors; and
memory with instructions that, if executed by the one or more processors, cause the system to:
for a particular load balancer, use a predictive model to determine a capacity useable in modifying a set of computing resources of the particular load balancer, where the predictive model is based at least on a configuration of a set of computer systems associated with the particular load balancer;
determine to modify the particular load balancer based at least in part on the configuration; and
as a result of the determination, modify the set of computing resources of the particular load balancer to the capacity.

6. The system of claim 5 , wherein the configuration of the set of computer systems includes at least one of the following:
process power of at least one computer system of the set of computer systems;
network bandwidth of the at least one computer system of the set of computer systems;
memory of the at least one computer system of the set of computer systems; and
a number of connections of the at least one computer system of the set of computer systems.

7. The system of claim 5 , wherein the memory further includes instructions that, if executed by the one or more processors, cause the system to:
determine a rate to modify the set of computing resources of the particular load balancer to the capacity; and
modify the set of computing resources of the particular load balancer to the capacity at the determined rate.

8. The system of claim 5 , wherein the memory further includes instructions that, if executed by the one or more processors, cause the system to generate the predictive model based at least in part on traffic patterns of the set of computer systems over an interval of time.

9. The system of claim 5 , wherein the instructions that cause the computer system to determine to modify the particular load balancer further include instructions that cause the computer system to determine to modify the particular load balancer in response to a request from a customer to modify the configuration.

10. The system of claim 5 , wherein the instructions that cause the computer system to modify the set of computing resources of the particular load balancer further include instructions that cause the computer system to instantiate an additional load balancer node with an additional capacity corresponding to the capacity.

11. The system of claim 5 , wherein the instructions that cause the computer system to modify the set of computing resources of the particular load balancer further include instructions that cause the computer system to assign one or more cached load balancer nodes to the particular load balancer.

12. The system of claim 11 , wherein assigning the one or more cached load balancer nodes to the particular load balancer further includes updating an entry in a naming system service associated with the load balancer.

13. A non-transitory computer-readable storage medium having stored thereon executable instructions that, if executed by one or more processors of a computer system, cause the computer system to:
generate a model of a capacity requirement of a load balancer based at least in part on a configuration of a set of servers for which the load balancer is responsible for distributing traffic to;
track the configuration of the set of servers;
use the model to determine an amount of capacity by which to adjust the load balancer based at least in part on the tracked configuration; and
adjust the load balancer by the determined amount of capacity.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the executable instructions that cause the computer system to track the configuration of the set of servers further include instructions that cause the computer system to track utilization of at least one of the following: network bandwidth; number of processors; memory usage; storage space; number of network interfaces; number of distributed servers; type of traffic received by at least one server of the set of servers; and utilization one or more servers of the set of servers.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the adjusting the load balancer by the determined amount of capacity comprises adding computing resources to the load balancer.

16. The non-transitory computer-readable storage medium of claim 13 , wherein the adjusting the load balancer by the determined amount of capacity comprises removing computing resources from a set of computing resources allocated to the load balancer.

17. The non-transitory computer-readable storage medium of claim 13 , wherein the executable instructions further comprise instructions that, if executed by the one or more processors, cause the computer system to:
track a rate of traffic directed to the set of servers over an interval of time; and
update the model based at least in part on the rate of traffic.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the executable instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to use the updated model to determine the amount of capacity to adjust the load balancer.

19. The non-transitory computer-readable storage medium of claim 13 , wherein the executable instructions further comprise instructions that, if executed by the one or more processors, cause the computer system to:
receive an application programming interface call configured to modify the configuration of the set of servers by at least adding an additional server to

Assignees

• Amazon Tech Inc

Inventors

• Young Lawrence Shao-Shien
• Thomas Christopher Carson
• Hashimoto Koji

Classifications

• G06N20/20

  Ensemble learning · CPC title

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/5083 Primary

  Techniques for rebalancing the load in a distributed system · CPC title