Dynamic virtual resource request rate control for utilizing physical resources

Patent metadata

Field	Value
Publication number	US-9635103-B2
Application number	US-201414483952-A
Country	US
Kind code	B2
Filing date	Sep 11, 2014
Priority date	Sep 11, 2014
Publication date	Apr 25, 2017
Grant date	Apr 25, 2017

Abstract

A virtualization host may implement dynamic virtual resource request rate controls for physical resources. Individual virtual resource request queues may be maintained for different virtual compute instances implemented at a virtualization host for a particular physical computer resource. After placing a work request from one of the individual virtual resource request queues into a physical resource request queue to be performed at the physical computer resource, a delay may be dynamically determined based, at least in part, on the workload of the physical resource request queue. After imposing the delay, a next work request from the individual virtual resource request queue may be placed into the physical resource request queue. In at least some embodiments, the dynamically determined delay may include a randomly added delay.

First claim

What is claimed is:

1. A system, comprising:
at least one processor;
a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to implement a virtualization host for a plurality of virtual compute instances;
wherein the virtualization host is configured to:
maintain a plurality of individual virtual resource request queues for respective virtual computer resources of the plurality of virtual compute instances that utilize a physical computer resource;
implement a dynamic rate control for individual ones of the plurality of individual virtual resource request queues;

the dynamic rate control, configured to:
place a work request for a virtual computer resource of a virtual compute instance from an individual virtual resource request queue into a physical resource request queue to perform the work request at the physical computer resource;
in response to the placement of the work request:
dynamically determine a delay based, at least in part, on a workload of the physical resource request queue; and
after imposition of the delay, place a next work request from the individual virtual resource request queue into the physical resource request queue;

wherein a work request from at least one other individual virtual resource request queue of the plurality of individual virtual resource request queues is placed in the physical resource request queue during the delay.

2. The system of claim 1 , wherein, to dynamically determine the delay, the dynamic rate control is configured to:
identify an initial delay for the individual virtual resource request queue based, at least in part, on a determined utilization of the physical computer resource for the virtual compute instance;
calculate a probability for adding a random delay based, at least in part, on the workload of the physical resource request queue; and
add the random delay to the initial delay according to the calculated probability.

3. The system of claim 2 , wherein the determined utilization of the physical computer resource for the virtual compute instance is based, at least in part, on a resource credit balance for the physical resource maintained for the virtual compute instance.

4. The system of claim 1 , wherein the virtualization host is implemented as part of a provider network that offers a network-based virtual computing service, wherein the virtualization host is multi-tenant such that at least one of the plurality of virtual compute instances implemented at the virtualization host is maintained for a client of the provider network that is different than another client of the provider network maintaining another one of the plurality of virtual compute instances at the virtualization host.

5. A method, comprising:
performing, by one or more computing devices that together implement a virtualization host for a plurality of virtual compute instances:
placing a work request for a virtual computer resource from an individual virtual resource request queue maintained for a virtual compute instance into a physical resource request queue to perform the work request at a physical computer resource, wherein the virtual compute instance is one of the plurality virtual compute instances, wherein the individual virtual resource request queue is one of a plurality of respective individual virtual resource request queues for respective virtual computer resources including the virtual computer resource of the plurality of virtual compute instances;
in response to placing the work request:
dynamically determining a delay based, at least in part, on a workload of the physical resource request queue; and
after imposing the delay, placing a next work request from the individual virtual resource request queue into the physical resource request queue;

wherein a work request from at least one other individual virtual resource request queue of the plurality of individual virtual resource request queues is placed in the physical resource request queue during the delay.

6. The method of claim 5 , wherein dynamically determining the delay comprises:
identifying an initial delay for the individual virtual resource request queue based, at least in part, on a determined utilization of the physical computer resource for the virtual compute instance;
calculating a probability for adding a random delay based, at least in part, on the workload of the physical resource request queue; and
adding the random delay to the initial delay according to the calculated probability.

7. The method of claim 6 , wherein the determined utilization of the physical computer resource for the virtual compute instance is based, at least in part, on a resource credit balance for the physical resource maintained for the virtual compute instance.

8. The method of claim 6 , wherein determining the workload of the physical resource request queue comprises smoothing one or more workload metrics indicating workload of the physical resource request queue at different points in time.

9. The method of claim 6 , further comprising:
in response to placing the next work request into the physical resource request queue:
performing dynamically determining the delay, wherein according to the calculated probability the random delay is not added to the initial delay; and
imposing the initial delay prior to placing another work request from the individual virtual resource request queue into the physical resource request queue.

10. The method of claim 5 , wherein placing the work request, dynamically determining the delay, and placing the next work request are performed for other individual virtual resource request queues for different virtual computer resources of the plurality of virtual compute instances that correspond to different physical computer resources.

11. The method of claim 5 , wherein the different physical computer resources comprise:
processing resources;
networking resources; or
storage resources.

12. The method of claim 5 , wherein a determined utilization of the physical computer resource for the virtual compute instance is higher than a determined utilization of the physical computer resource for a virtual compute instance for which the work request from the at least one other individual virtual resource request queue is placed in the physical resource request queue during the delay.

13. The method of claim 5 , wherein the virtualization host is implemented as part of a provider network that offers a network-based virtual computing service, wherein the virtualization host is multi-tenant such that at least one of the plurality of virtual compute instances implemented at the virtualization host is maintained for a client of the provider network that is different than another client of the provider network maintaining another one of the plurality of virtual compute instances at the virtualization host.

14. A non-transitory computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement a virtualization host for a plurality of compute instances, wherein the virtualization host implements:
placing a work request for a virtual computer resource from an individual virtual resource request queue maintained for a virtual compute instance into a physical resource request queue to perform the work request at a physical computer resource, wherein the virtual compute instance is one of the plurality virtual compute instances, wherein the individual virtual resource request queue is one of a plurality of respective individual virtual resourc

Assignees

• Amazon Tech Inc

Inventors

• Earl William John

Classifications

• H04L67/1031 Primary

  Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests · CPC title

• G06F2209/5013

  Request control · CPC title

• G06F9/5016

  the resource being the memory · CPC title

• G06F2209/5019

  Workload prediction · CPC title

• H04L67/1008

  based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) · CPC title