Systems and methods for dynamic load balancing based on server utilization and content popularity

Patent metadata

Field	Value
Publication number	US-11451623-B2
Application number	US-202117329981-A
Country	US
Kind code	B2
Filing date	May 25, 2021
Priority date	Oct 26, 2020
Publication date	Sep 20, 2022
Grant date	Sep 20, 2022

Abstract

Provided is a controller for dynamically balancing load between different servers using different thresholds that are continually modified for each of the servers. The controller may generate a baseline load measure based on load measures received from the different servers, and may configure a first threshold for a first server and a second threshold for a second server based on the load measure at the first server deviating from the baseline load measure by a first amount that is greater than a second amount by which the load measure at the second server deviates from the baseline load measure. The controller may allocate an additional server to distribute first content with the first server in response to first content load at the first server satisfying the first threshold and the same load or a greater load of second content at the second server not satisfying the second threshold.

First claim

What is claimed is:

1. A method comprising:
measuring a first amount of load experienced by a first device, wherein the first device distributes a first set of content over a network from a particular site;
measuring a second amount of load experienced by a second device, wherein the second device distributes a second set of content over the network from the particular site;
determining an amount of deviation between the first amount of load and the second amount of load; and
setting different thresholds at which additional resources are allocated to (i) distribute one or more of the first set of content in conjunction with the first device or (ii) distribute the second set of content in conjunction with the second device in response to the amount of deviation.

2. The method of claim 1 ,
wherein measuring the first amount of load comprises tracking one or more of a first number of requests for the first set of content received at the first device or data served in response to the first number of requests; and
wherein measuring the second amount of load comprises tracking one or more of a second number of requests for the second set of content received at the second device or data served in response to the second number of requests.

3. The method of claim 1 further comprising:
detecting an uneven distribution of load on the first device in response to the first amount of load being greater than the second amount of load;
determining first content from the first set of content imposing at least a particular amount of load on the first device, and second content from the second set of content imposing at least the particular amount of load on the second device; and
allocating resources from a third device or the second device at the particular site to distribute the first content in conjunction with the first device while retaining distribution of the second content entirely within the second device in response to setting the different thresholds.

4. The method of claim 3 , wherein setting the different thresholds comprises:
configuring a first threshold for the first device, wherein the first threshold is satisfied by the particular amount of load; and
configuring a second threshold for the second device, wherein the second threshold is larger than the first threshold and is not satisfied by the particular amount of load.

5. The method of claim 1 further comprising:
computing overall load for the particular site based on load experienced by a plurality of devices at the particular site, wherein the plurality of devices comprises the first device and the second device; and
wherein the determining the amount of deviation comprises computing a first deviation between the first amount of load and the overall load, and a second deviation between the second amount of load and the overall load.

6. The method of claim 1 further comprising:
tracking an individual contribution of load from each content of the first set of content to the first amount of load;
comparing the individual contribution of load from each content of the first set of content against a first threshold that is configured for the first device as a result of setting the different thresholds; and
allocating an additional device or resource of the particular site to distribute first content from the first set of content in conjunction with the first device in response to the individual contribution of load from the first content satisfying or exceeding the first threshold.

7. The method of claim 6 further comprising:
distributing second content from the first set of content exclusively from the first device in response to the individual contribution of load from the second content not satisfying or exceeding the first threshold.

8. The method of claim 6 further comprising:
determining a number of additional resources or devices to allocate for distribution of the first content based on a value by which the individual contribution of load from the first content exceeds the first threshold.

9. The method of claim 6 , wherein allocating the additional device or resource comprises:
modifying a configuration of a request director for the particular site, wherein the request director receives a plurality of content requests that are directed to the first content, and the request director distributes the plurality of content requests across the first device and the additional device or resource.

10. The method of claim 6 further comprising:
selecting the additional device or resource from a plurality of devices or resources of the particular site based on the additional device or resource being a least loaded device or resource of the plurality of devices or resources; and
shifting part of the individual contribution of load of the first content from the first device to the additional device or resource.

11. The method of claim 1 further comprising:
distributing first content from the first set of content with the first device and an additional resource or device of the particular site in response to an individual load from the first content on the first device exceeding a first threshold that is configured for the first device as a result of setting the different thresholds; and
distributing second content from the second set of content with the second device and an additional resource or device of the particular site in response to an individual load from the second content on the second device exceeding a second threshold that is configured for the second device as a result of setting the different thresholds, and wherein the first threshold is different than the second threshold.

12. The method of claim 11 , wherein a difference between the first threshold and the second threshold is based on the amount of deviation between the first amount of load and the second amount of load.

13. The method of claim 11 , wherein a difference between the first threshold and the second threshold is based on a difference between the first amount of load and an average load at the particular site, and a difference between the second amount of load and the average load at the particular site.

14. The method of claim 1 further comprising:
tracking an individual load imposed on the first device by each content of the first set of content; and
calculating the first amount of load based on the individual load imposed by each content of the first set of content.

15. The method of claim 1 further comprising:
scaling resources that are allocated to distribute first content from the first set of content in response to load contributed by the first content on the first device exceeding a first threshold configured for the first device as a result of setting the different thresholds; and
scaling resources that are allocated to distribute second content from the second set of content in response to load contributed by the second content on the second device exceeding a different second threshold configured for the second device as a result of setting the different thresholds.

16. A system comprising:
a particular site with at least a first device that distributes a first set of content over a network, and a second device that distributes a second set of content over the network; and
one or more processors configured to:
measure a first amount of load experienced by the first device;
measure a second amount of load experienced by the second device;
determine an amount of deviation between the first amount of load and the second amount of load; and
set different thresholds at which additional resources are allocated to (

Assignees

• Verizon Digital Media Services Inc
• Edgecast Inc

Inventors

• Zarifis Kyriakos
• Bedi Harkeerat Singh
• Yeager Donnevan Scott
• Shiell Derek

Classifications

• H04L67/1012

  based on compliance of requirements or conditions with available server resources · CPC title

• H04L67/1029

  using data related to the state of servers by a load balancer · CPC title

• H04L67/1008 Primary

  based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) · CPC title

• H04L67/1031

  Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests · CPC title