Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment

Patent metadata

Field	Value
Publication number	US-11153371-B2
Application number	US-201916568149-A
Country	US
Kind code	B2
Filing date	Sep 11, 2019
Priority date	Dec 10, 2010
Publication date	Oct 19, 2021
Grant date	Oct 19, 2021

Abstract

Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment. According to an example, an application server employs a traffic light metaphor to represent a utilization level of resources of the application server by associating a traffic light with each resource. A mapping is maintained that associates service requests with corresponding sets of affected traffic lights. A deferred queue is maintained for each traffic light to facilitate throttling of service requests directed to the application server that involve a resource that is under pressure. Responsive to receiving a service request directed to the application server, the service request is added directly or indirectly to one of multiple queues maintained in front of the application server based on a priority associated with the service request. Service requests are serviced from the queues in accordance with a priority associated with the queues.

First claim

What is claimed is:

1. A method comprising:
employing, by one or more processors of an application server of a plurality of application servers, a traffic light metaphor to represent a utilization level of each of a plurality of resources of the application server by associating a traffic light of a plurality of traffic lights with each resource of the plurality of resources;
maintaining, by the one or more processors, a mapping that associates a particular service request directed to the application server with a set of affected traffic lights of the plurality of traffic lights;
facilitating, by the one or more processors, throttling of service requests directed to the application server that involve a resource of the plurality of resources that is under pressure by maintaining a deferred queue of a plurality of deferred queues for each traffic light of the plurality of traffic lights;
responsive to receiving, by the one or more processors, a service request directed to the application server, adding directly or indirectly, based on a priority associated with the service request, the service request to a queue of a plurality of queues maintained in front of the application server; and
servicing, by the one or more processors, service requests from the plurality of queues in accordance with a priority associated with each of the plurality of queues.

2. The method of claim 1 , further comprising:
determining, by the queue, states of the set of affected traffic lights for the service request based on the mapping;
when a state of one or more traffic lights of the set of affected traffic lights for the service request is red, postponing the service request by directly or indirectly placing the service request on the deferred queue for a first traffic light of the one or more traffic lights; and
when no state of any of the set of affected traffic lights for the service request is red, enqueueing the service request on the queue.

3. The method of claim 2 , further comprising:
maintaining, by the one or more processors, a directory of utilization information received from other of the plurality of application servers, wherein the directory facilitates identification of those of the other application servers of the plurality of application servers that are in a healthy state; and
when a given deferred queue of the plurality of deferred queues meets a first handoff threshold or a particular service request on the given deferred queue meets a second handoff threshold, then removing the particular service request from the given deferred queue and forwarding the particular service request to one of the other application servers that is in the healthy state.

4. The method of claim 3 , wherein the first handoff threshold comprises a predefined queue depth.

5. The method of claim 3 , wherein the second handoff threshold comprises an expiration time/date.

6. The method of claim 2 , further comprising:
responsive to detecting, by the one or more processors, a state of a particular traffic light of the plurality of traffic lights is no longer red, pulling a next service request from a particular deferred queue of the plurality of deferred queues associated with the particular traffic light;
determining, by the one or more processors, states of the set of affected traffic lights for the next service request based on the mapping;
when a state of one or more traffic lights of the set of affected traffic lights for the next service request is red, continuing to postpone the next service request; and
when no state of any of the set of affected traffic lights for the next service request is red, adding directly or indirectly the next service request to one of the plurality of queues maintained in front of the application server.

7. The method of claim 1 , wherein said adding directly or indirectly, based on a priority associated with the service request, the service request to a queue of a plurality of queues maintained in front of the application server comprises adding an object associated with the service request to the queue.

8. The method of claim 7 , wherein the object encapsulates metadata of the service request.

9. The method of claim 8 , wherein the priority associated with the service request is determined based on the metadata of the service request.

10. The method of claim 1 , wherein the plurality of resources include a connection pool, virtual machine memory, input/output (I/O) bandwidth, and processors.

11. An article comprising a non-transitory computer-readable medium having stored thereon instructions that when executed by one or more processors of an application server of a plurality of application servers are configurable to service requests by:
employing a traffic light metaphor to represent a utilization level of each of a plurality of resources of the application server by associating a traffic light of a plurality of traffic lights with each resource of the plurality of resources;
maintaining a mapping that associates a particular service request directed to the application server with a set of affected traffic lights of the plurality of traffic lights;
facilitating throttling of service requests directed to the application server that involve a resource of the plurality of resources that is under pressure by maintaining a deferred queue of a plurality of deferred queues for each traffic light of the plurality of traffic lights;
responsive to receiving a service request directed to the application server, adding directly or indirectly, based on a priority associated with the service request, the service request to a queue of a plurality of queues maintained in front of the application server; and
servicing service requests from the plurality of queues in accordance with a priority associated with each of the plurality of queues.

12. The article of claim 11 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
determine states of the set of affected traffic lights for the service request based on the mapping;
when a state of one or more traffic lights of the set of affected traffic lights for the service request is red, postpone the service request by directly or indirectly placing the service request on the deferred queue for a first traffic light of the one or more traffic lights; and
when no state of any of the set of affected traffic lights for the service request is red, enqueue the service request on the queue.

13. The article of claim 12 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
maintain a directory of utilization information received from other of the plurality of application servers, wherein the directory facilitates identification of those of the other application servers of the plurality of application servers that are in a healthy state; and
when a given deferred queue of the plurality of deferred queues meets a first handoff threshold or a particular service request on the given deferred queue meets a second handoff threshold, remove the particular service request from the given deferred queue and forward the particular service request to one of the other application servers that is in the healthy state.

14. The article of claim 12 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
responsive to detecting a state of a particular traffic light of the plurality of traffic lights is no longer red, pull a next service request from a particular deferred queue of the plurality of deferred que

Assignees

• Salesforce Com Inc

Inventors

• Yang Ronald
• Devadhar Vijayanth
• Cheenath Manoj

Classifications

• H04L49/90

  Buffering arrangements · CPC title

• H04L67/1008 Primary

  based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• H04L69/329

  in the application layer [OSI layer 7] · CPC title

• H04L67/1001

  for accessing one among a plurality of replicated servers · CPC title