Asynchronous task management in an on-demand network code execution environment

Patent metadata

Field	Value
Publication number	US-9952896-B2
Application number	US-201615195897-A
Country	US
Kind code	B2
Filing date	Jun 28, 2016
Priority date	Jun 28, 2016
Publication date	Apr 24, 2018
Grant date	Apr 24, 2018

Abstract

Systems and methods are described for managing asynchronous code executions in an on-demand code execution system or other distributed code execution environment, in which multiple execution environments, such as virtual machine instances, can be used to enable rapid execution of user-submitted code. When asynchronous executions occur, one execution may become blocked while waiting for completion of another execution. Because the on-demand code execution system contains multiple execution environments, the system can efficiently handle a blocked execution by saving a state of the execution, and removing it from its execution environment. When a blocking dependency operation completes, the system can resume the blocked execution using the state information, in the same or different execution environment.

First claim

What is claimed is:

1. A system to manage blocking of code executions in an on-demand code execution system due to asynchronous operations, wherein the on-demand code execution system comprises a plurality of execution environments on which user-submitted code may execute, the system comprising:
a non-transitory data store configured to store state information regarding suspended executions of tasks on the on-demand code execution system, wherein individual tasks are associated with code executable to implement functionality corresponding to the individual tasks; and
one or more processors configured with computer-executable instructions to:
obtain instructions to execute a first task associated with first executable code;
begin execution of the first executable code within a first execution environment, wherein execution of the first executable code calls for execution of a first dependency operation;
after detecting that execution of the first executable code has become blocked awaiting completion of the first dependency operation:
determine that a predicted duration of blocking for the execution of the first executable code satisfied a threshold value;
generate state information for the execution of the first executable code;
store the generated state information in the non-transitory data store; and
remove the execution of the first executable code from the first execution environment; and

after detecting that the first dependency operation has completed:
select a second execution environment in which to resume execution of the first executable code; and
utilize the generated state information, as stored in the non-transitory data store, to resume execution of the first executable code in the second execution environment.

2. The system of claim 1 , wherein the first execution environment is at least one of a virtual machine instance or a container.

3. The system of claim 1 , wherein the second execution environment is a different execution environment from the first execution environment.

4. The system of claim 1 , wherein the second execution environment is a regenerated version of the first execution environment.

5. The system of claim 1 , wherein the state information for the execution of the first executable code comprises at least one of a virtual machine state, a container state, a state of memory associated with the execution, or a state of objects of the first executable code during the execution.

6. A computer-implemented method to manage blocking of code executions in an on-demand code execution system, wherein the on-demand code execution system comprises a plurality of execution environments on which user-submitted code may execute, the computer-implemented method comprising:
obtain instructions to execute first executable code within a first execution environment of the on-demand code execution system;
after detecting that execution of the first executable code is blocked awaiting completion of a first dependency operation:
generating state information for the execution of the first executable code;
storing the generated state information in a non-transitory data store distinct from the first execution environment; and
removing the execution of the first executable code from the first execution environment; and

after detecting that the first dependency operation has completed:
selecting a second execution environment in which to resume execution of the first executable code; and
utilizing the generated state information, as stored in the non-transitory data store, to resume execution of the first executable code in the second execution environment.

7. The computer-implemented method of claim 6 further comprising:
predicting a duration of blocking for the execution of the first executable code; and
determining that the predicted duration of blocking for the execution of the first executable code satisfied a threshold value.

8. The computer-implemented method of claim 7 , wherein predicting a duration of blocking for the execution of the first executable code comprises:
predicting a first length of time between when the execution of the first executable code called for execution of the first dependency operation and when the execution of the first executable code requires completion of the first dependency operation;
predicting a second length of time required for execution of the first dependency operation to complete; and
assigning a difference between the first length of time and the second length of time as the predicted duration of blocking for the execution of the first executable code.

9. The computer-implemented method of claim 8 , wherein the first length of time is predicted based on historical executions of the first executable code.

10. The computer-implemented method of claim 8 , wherein the second length of time is predicted based on historical executions of the first dependency operation.

11. The computer-implemented method of claim 6 , wherein selecting a second execution environment in which to resume execution of the first executable code comprises selecting the second execution environment based at least in part on the state information.

12. The computer-implemented method of claim 6 , wherein the execution of the first executable code causes a call to execute an instance of the first dependency operation, and wherein detecting that the first dependency operation has completed comprises detecting that the instance of the first dependency operation has completed.

13. The computer-implemented method of claim 6 , wherein a plurality of executions of the first executable code depend on completion of the first dependency operation, and wherein the method further comprises, on detecting that the first dependency operation has completed, selecting at least one of the plurality of executions of the first executable code to be resumed.

14. The computer-implemented method of claim 6 further comprising:
during execution of the first execution code, detecting a call for execution of a second dependency operation;
determining a deadline for the second dependency operation based at least in part on historical data regarding prior executions of the first executable code;
enqueuing the first dependency operation into the queue based at least in part on the deadline; and
processing the queue based at least in part on an available capacity of the on-demand code execution system to execute operations, wherein processing the queue comprises executing the first dependency operation.

15. Non-transitory computer-readable storage media including computer-executable instructions that, when executed by a computing system, cause the computing system to:
obtain instructions to execute first executable code within a first execution environment of an on-demand code execution system comprising a plurality of execution environments;
after detecting that execution of the first executable code has become blocked awaiting completion of a first dependency operation:
generate state information for the execution of the first executable code;
store the generated state information in a non-transitory data store distinct from the first execution environment; and
remove the execution of the first executable code from the first execution environment; and
after detecting that the first dependency operation has completed:
select a second execution environment in which to resume execution of the first executable code; and
utilize the generated state information, as stored in the non-transitory data store, to resume execution of the first executable code in the second execution

Assignees

• Amazon Tech Inc

Inventors

• Wagner Timothy Allen
• Brooker Marc John
• Nair Ajay

Classifications

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/4831 Primary

  with variable priority · CPC title

• G06F9/485

  Task life-cycle, e.g. stopping, restarting, resuming execution (G06F9/4881 takes precedence) · CPC title

• G06F9/52 Primary

  Program synchronisation; Mutual exclusion, e.g. by means of semaphores · CPC title