Adjusting variable limit on concurrent code executions

Patent metadata

Field	Value
Publication number	US-2019050271-A1
Application number	US-201816160674-A
Country	US
Kind code	A1
Filing date	Oct 15, 2018
Priority date	Jun 29, 2016
Publication date	Feb 14, 2019
Grant date	—

Abstract

Systems and methods are described for adjusting a number of concurrent code executions allowed to be performed for a given user on an on-demand code execution environment or other distributed code execution environments. Such environments utilize pre-initialized virtual machine instances to enable execution of user-specified code in a rapid manner, without delays typically caused by initialization of the virtual machine instances. However, to improve utilization of computing resources, such environments may temporarily restrict the number of concurrent code executions performed on behalf of the given user to a number less than the maximum number of concurrent code executions allowed for the given user. Such environments may adjust the temporary restriction on the number of concurrent code executions based on the number of incoming code execution requests associated with the given user.

First claim

What is claimed is:

1 . A system comprising:
one or more processors; one or more memories, the one or more memories having stored thereon instructions, which, when executed by the one or more processors, cause the one or more processors to:
receive a request to execute a program code, the request including an identifier to be used to acquire compute capacity;
identify, using the identifier, a first virtual machine instance manager from a set of virtual machine instance managers configured to acquire compute capacity;
request compute capacity from the first virtual machine instance manager;
determine that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity;
cause a second virtual machine instance manager to be added to the set, the second virtual machine instance manager being different from the first virtual machine instance manager;
request compute capacity from the second virtual machine instance manager; and
cause the program code to be executed using the compute capacity acquired by the second virtual machine instance manager.

2 . The system of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to determine, prior to causing the second virtual machine instance manager to be added to the set, that a condition for adding an additional virtual machine instance manager to the set is satisfied.

3 . The system of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
calculate a hash value by applying a hash function to the identifier included in the request; and select the first virtual machine instance manager corresponding to the calculated hash value.

4 . The system of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
access a cache indicating the set of virtual machine instance managers associated with the identifier; and select the first virtual machine instance manager from the set of virtual machine instance managers.

5 . The system of claim 1 , wherein the virtual machine instance managers in the set are arranged in a specific order, the instructions, when executed by the one or more processors, further cause the one or more processors to:
receive another request to execute the program code, said another request including the identifier; and request compute capacity from a next virtual machine instance manager in the set that immediately follows the second virtual machine instance manager in the specific order.

6 . The system of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to cause the second virtual machine instance manager to be removed from the set in response to determining that a condition for removing a virtual machine instance manager from the set is satisfied.

7 . The system of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
in response to determining that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity, request load information associated with the identifier from another frontend module associated with another set of virtual machine instance managers; and cause a number of virtual machine instance managers in said another set to be adjusted based on the load information.

8 . A computer-implemented method comprising:
receiving a request to execute a program code, the request including an identifier to be used to acquire compute capacity; identifying, using the identifier, a first virtual machine instance manager from a set of virtual machine instance managers configured to acquire compute capacity; requesting compute capacity from the first virtual machine instance manager; determining that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity; adding a second virtual machine instance manager to the set, the second virtual machine instance manager being different from the first virtual machine instance manager; requesting compute capacity from the second virtual machine instance manager; and executing the program code using the compute capacity acquired by the second virtual machine instance manager.

9 . The computer-implemented method of claim 5 , further comprising determining, prior to adding the second virtual machine instance manager to the set, that a condition for adding an additional virtual machine instance manager to the set is satisfied.

10 . The computer-implemented method of claim 5 , wherein identifying the first virtual machine instance manager comprises:
calculating a hash value by applying a hash function to the identifier included in the request; and selecting the first virtual machine instance manager corresponding to the calculated hash value.

11 . The computer-implemented method of claim 5 , wherein identifying the first virtual machine instance manager comprises:
accessing a cache indicating the set of virtual machine instance managers associated with the identifier; and selecting the first virtual machine instance manager from the set of virtual machine instance managers.

12 . The computer-implemented method of claim 5 , wherein the virtual machine instance managers in the set are arranged in a specific order, the method further comprising:
receiving another request to execute the program code, said another request including the identifier; and requesting compute capacity from a next virtual machine instance manager in the set that immediately follows the second virtual machine instance manager in the specific order.

13 . The computer-implemented method of claim 5 , further comprising, in response to determining that a condition for removing a virtual machine instance manager from the set is satisfied, removing the second virtual machine instance manager from the set.

14 . The computer-implemented method of claim 5 , further comprising:
in response to determining that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity, requesting load information associated with the identifier from another frontend module associated with another set of virtual machine instance managers; and adjusting a number of virtual machine instance managers in said another set based on the load information.

15 . Non-transitory physical computer storage including computer-executable instructions that, when executed by a computing system, cause the computing system to:
receive a request to execute a program code, the request including an identifier to be used to acquire compute capacity; identify, using the identifier, a first virtual machine instance manager from a set of virtual machine instance managers configured to acquire compute capacity; request compute capacity from the first virtual machine instance manager; determine that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity; cause a second virtual machine instance manager to be added to the set, the second virtual machine instance manager being different from the first virtual machine instance manager; request compute capacity from the second virtual machine instance manager; and cause the program code to be executed using the compute capacity acquired by the second virtual machine instance manager.

Assignees

• Amazon Tech Inc

Inventors

• Marriner Dylan Owen
• Roman Mauricio
• Brooker Marc John
• Herwitz Julian Embry
• Reque Sean

Classifications

• G06F9/5077 Primary

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F2009/4557

  Distribution of virtual machine instances; Migration and load balancing · CPC title

• G06F9/45558

  Hypervisor-specific management and integration aspects · CPC title

• G06F2209/504

  Resource capping · CPC title

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title