Adjusting variable limit on concurrent code executions

Patent metadata

Field	Value
Publication number	US-11354169-B2
Application number	US-201916552535-A
Country	US
Kind code	B2
Filing date	Aug 27, 2019
Priority date	Jun 29, 2016
Publication date	Jun 7, 2022
Grant date	Jun 7, 2022

Abstract

Systems and methods are described for adjusting a number of concurrent code executions allowed to be performed for a given user on an on-demand code execution environment or other distributed code execution environments. Such environments utilize pre-initialized virtual machine instances to enable execution of user-specified code in a rapid manner, without delays typically caused by initialization of the virtual machine instances. However, to improve utilization of computing resources, such environments may temporarily restrict the number of concurrent code executions performed on behalf of the given user to a number less than the maximum number of concurrent code executions allowed for the given user. Such environments may adjust the temporary restriction on the number of concurrent code executions based on the number of incoming code execution requests associated with the given user.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more memories, the one or more memories having stored thereon instructions, which, when executed by the one or more processors, configure the one or more processors to:
receive a plurality of requests to execute a program code, wherein the program code is associated with a user;
determine load information associated with a set of virtual machine instance managers associated with the user, wherein the set of virtual machine instance managers are each configured to acquire compute capacity to be used to execute the program code on behalf of the user, and wherein the set of virtual machine instance managers includes at least two virtual machine instance managers such that requests to acquire compute capacity for at least some of the plurality of requests are distributable across the at least two virtual machine instance managers; and
cause, based on the load information satisfying a threshold condition, one or more virtual machine instance managers in the set of virtual machine instance managers to be removed from the set of virtual machine instance managers associated with the user such that the one or more removed virtual machine instance managers are no longer requested to acquire compute capacity to be used to execute the program code on behalf of the user.

2. The system of claim 1 , wherein the instructions, when executed by the one or more processors, further configure the one or more processors to request the set of virtual machine instance managers associated with the user to each report back its load information.

3. The system of claim 1 , wherein satisfying the threshold condition comprises the load information indicating that an average load value of the set of virtual machine instance managers is below a threshold load value.

4. The system of claim 1 , wherein satisfying the threshold condition comprises the load information indicating that a request to execute the program code has not been received for a threshold amount of time.

5. The system of claim 1 , wherein the instructions, when executed by the one or more processors, further configure the one or more processors to periodically check whether an average load across all of the set of virtual machine instance managers associated with the user is below a threshold level, and based on the average load being below the threshold level, cause one or more additional virtual machine instance managers to be removed from the set.

6. The system of claim 1 , wherein the instructions, when executed by the one or more processors, further configure the one or more processors to distribute incoming code execution requests associated with the user across the set of virtual machine instance managers.

7. The system of claim 1 , wherein the set of virtual machine instance managers are mapped onto different hash values on a ring of hash values such that each hash value on the ring is assigned to one of the virtual machine instance managers in the set,
wherein the instructions, when executed by the one or more processors, further configure the one or more processors to determine a hash value associated with an incoming request to execute the program code, and cause the incoming request to be routed to a specific virtual machine instance manager in the set that is assigned to the determined hash value.

8. The system of claim 7 , wherein the instructions, when executed by the one or more processors, further configure the one or more processors to cause one or more hash values associated with the one or more removed virtual machine instance managers to be associated with one or more remaining virtual machine instance managers in the set.

9. A computer-implemented method comprising:
receiving a plurality of requests to execute a program code, wherein the program code is associated with a user;
determining load information associated a set of virtual machine instance managers associated with the user, wherein the set of virtual machine instance managers are each configured to acquire compute capacity to be used to execute the program code on behalf of the user, and wherein the set of virtual machine instance managers includes at least two virtual machine instance managers such that requests to acquire compute capacity for at least some of the plurality of requests are distributable across the at least two virtual machine instance managers; and
removing, based on the load information satisfying a threshold condition, one or more virtual machine instance managers in the set of virtual machine instance managers from the set of virtual machine instance managers associated with the user such that the one or more removed virtual machine instance managers are no longer requested to acquire compute capacity to be used to execute the program code on behalf of the user.

10. The computer-implemented method of claim 9 , further comprising requesting the set of virtual machine instance managers associated with the user to each report back its load information.

11. The computer-implemented method of claim 9 , wherein satisfying the threshold condition comprises the load information indicating that an average load value of the set of virtual machine instance managers is below a threshold load value.

12. The computer-implemented method of claim 9 , wherein satisfying the threshold condition comprises the load information indicating that a request to execute the program code has not been received for a threshold amount of time.

13. The computer-implemented method of claim 9 , further comprising distributing incoming code execution requests associated with the user across the set of virtual machine instance managers.

14. The computer-implemented method of claim 9 , wherein the set of virtual machine instance managers are mapped onto different hash values on a ring of hash values such that each hash value on the ring is assigned to one of the virtual machine instance managers in the set,
wherein the method further comprises determining a hash value associated with an incoming request to execute the program code, and routing the incoming request to a specific virtual machine instance manager in the set that is assigned to the determined hash value.

15. The computer-implemented method of claim 14 , further comprising associating one or more hash values associated with the one or more removed virtual machine instance managers with one or more remaining virtual machine instance managers in the set.

16. Non-transitory physical computer storage including computer-executable instructions that, when executed by a computing system, configure the computing system to:
receive a plurality of requests to execute a program code, wherein the program code is associated with a user;
determine load information associated a set of virtual machine instance managers associated with the user, wherein the set of virtual machine instance managers are each configured to acquire compute capacity to be used to execute the program code on behalf of the user, and wherein the set of virtual machine instance managers includes at least two virtual machine instance managers such that requests to acquire compute capacity for at least some of the plurality of requests are distributable across the at least two virtual machine instance managers; and
cause, based on the load information satisfying a threshold condition, one or more virtual machine instance managers in the set of virtual machine instance managers to be removed from the set of virtual machine instance managers associated with the user such that the one or more removed virt

Assignees

• Amazon Tech Inc

Inventors

• Marriner Dylan Owen
• Roman Mauricio
• Brooker Marc John
• Herwitz Julian Embry
• Reque Sean

Classifications

• G06F9/45558

  Hypervisor-specific management and integration aspects · CPC title

• G06F9/5077 Primary

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F2209/504

  Resource capping · CPC title

• G06F2009/4557

  Distribution of virtual machine instances; Migration and load balancing · CPC title

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title