Load controller framework

Patent metadata

Field	Value
Publication number	US-9270617-B2
Application number	US-201313910461-A
Country	US
Kind code	B2
Filing date	Jun 5, 2013
Priority date	Jun 5, 2013
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

The present disclosure involves systems, software, and computer-implemented methods for controlling service load in a cloud-based system. An example method includes receiving a first request for the network service from a client, evaluating a load condition associated with the network service, the load condition indicating an availability of the network service to receive requests, returning a unique token associated with the first request to the client in response to the load condition indicating that the network service is not available to receive the requests, receiving a second request for the network service from the client, the second request including at least a portion of the first request and the unique token, evaluating the load condition associated with the network service, and prioritizing the second request based on the unique token in response to the load condition indicating that the network service is available to receive the requests.

First claim

What is claimed is:

1. A computer-implemented method executed by one or more processors, the method performed at a load controller of a cloud-based network system, the method comprising:
receiving a first request for a network service from a client, wherein the load controller is separate from the network service;
evaluating a load condition associated with the network service, the load condition indicating an availability of the network service to receive requests, based at least in part on a determination of whether a flow control indication is received by the load controller from the network service indicating that the network service is currently unavailable to receive the requests due to load;
returning a unique token associated with the first request to the client in response to the load condition indicating that the network service is not available to receive the requests, wherein the unique token is associated with a unique identifier and wherein an entry to a token database at the load controller associated with the unique token is entered, the entry at the token database including a set of statistics associated with the token including a number of times the unique token has been submitted to the load controller;
after returning the unique token to the client, receiving a second request for the network service from the client, the second request including at least a portion of the first request and the unique token;
re-evaluating the load condition associated with the network service in response to receiving the second request;
prioritizing the second request based on the unique token in response to the load condition indicating that the network service is available to receive the requests, wherein prioritizing the second request includes placing the second request into a queue with a plurality of other requests; and
in response to receiving a flow control indication after prioritizing the second request and prior to fulfilling the second request, removing at least the second request from the queue and returning the unique token to the client.

2. The method of claim 1 , where prioritizing the second request based on the unique token includes sending the second request to the network service before a third request that was received prior to the second request.

3. The method of claim 1 , wherein evaluating the load condition associated with the network service further comprises:
analyzing current statistics associated with the availability of the network service; and
evaluating the statistics based at least in part on one or more rules, the one or more rules including thresholds associated with the availability of the network service.

4. The method of claim 3 , wherein the one or more rules each include one or more actions to be taken when the statistics associated with the availability of the network service indicate that the network service is not available.

5. The method of claim 4 , wherein the one or more actions include at least one of: reducing a rate that requests are sent to the network service for a period of time or blocking access to the network service for a period of time.

6. The method of claim 3 , wherein the thresholds each include a time value indicating a period of time that the threshold is to be used in evaluating the statistics.

7. The method of claim 3 , wherein the current statistics associated with the availability of the network service include at least one of a number of requests sent to the network service in a recent time period, a total amount of data sent to the network service in the recent time period, and a total number of outstanding requests that the network service is currently processing.

8. The method of claim 1 , wherein evaluating the load condition associated with the network service further comprises receiving an indication from the network service indicating the availability of the network service to receive requests.

9. The method of claim 1 , wherein the network service is a network service in the cloud-based network system.

10. The method of claim 1 , wherein the network service is a customer-premise network service separate from the cloud-based network system.

11. A computer program product encoded on a tangible, non-transitory storage medium, the product comprising computer readable instructions for causing one or more processors to perform operations comprising:
receiving, at a load controller of a cloud-based network system, a first request for a network service from a client, wherein the load controller is separate from the network service;
evaluating a load condition associated with the network service, the load condition indicating an availability of the network service to receive requests, based at least in part on a determination of whether a flow control indication is received by the load controller from the network service indicating that the network service is currently unavailable to receive the requests due to load;
returning a unique token associated with the first request to the client in response to the load condition indicating that the network service is currently unavailable to receive the requests, wherein the unique token is associated with a unique identifier and wherein an entry to a token database at the load controller associated with the unique token is entered, the entry at the token database including a set of statistics associated with the token including a number of times the unique token has been submitted to the load controller;
after returning the unique token to the client, receiving a second request for the network service from the client, the second request including at least a portion of the first request and the unique token;
re-evaluating the load condition associated with the network service in response to receiving the second request; and
prioritizing the second request based on the unique token in response to the load condition indicating that the network service is available to receive the requests, wherein prioritizing the second request includes placing the second request into a priority queue, the priority queue separate from a standard queue used for requests received without an included unique token, and wherein the priority queue is prioritized over the standard queue; and
in response to receiving a flow control indication after prioritizing the second request and prior to fulfilling the second request, removing at least the second request from the priority queue and returning the unique token to the client.

12. The computer program product of claim 11 , where prioritizing the second request based on the unique token includes sending the second request to the network service before a third request that was received prior to the second request.

13. The computer program product of claim 11 , wherein in response to determining that the network service is currently unavailable to receive the requests due to load, one or more rules are evaluated to determine one or more actions to be taken when the network service is currently unavailable, wherein the one or more actions include at least one of: reducing a rate that requests are sent to the network service for a period of time or blocking access to the network service for a period of time.

14. The computer program product of claim 11 , wherein the network service is a network service in the cloud-based network system.

15. The computer program product of claim 11 , wherein the network service is a customer-premise network service separate from the cloud-based network system.

16. A system, comprising:
memory for storing data; and
one or more pro

Assignees

• Schmidt Olaf
• Fischer Martin P
• Sap Se

Inventors

• Schmidt Olaf
• Fischer Martin P

Classifications

• H04L47/11

  Identifying congestion · CPC title

• G06F9/5083

  Techniques for rebalancing the load in a distributed system · CPC title

• H04L47/125

  by balancing the load, e.g. traffic engineering · CPC title

• H04L47/821 Primary

  Prioritising resource allocation or reservation requests · CPC title

• H04L47/627

  policing · CPC title