Reducing execution times in an on-demand network code execution system using saved machine states

Patent metadata

Field	Value
Publication number	US-11836516-B2
Application number	US-202117445699-A
Country	US
Kind code	B2
Filing date	Aug 23, 2021
Priority date	Jul 25, 2018
Publication date	Dec 5, 2023
Grant date	Dec 5, 2023

Abstract

Systems and methods are described for reducing latency to service requests to execute code on an on-demand code execution system by maintaining snapshots of virtual machine instances in a ready state to execute such code. A user may submit code to the on-demand code execution system, which code depends on other software, such as an operating system or runtime. The on-demand code execution system can generate a virtual machine instance provisioned with the other software, and initialize the instance into a state at which it is ready to execute the code. The on-demand code execution system can then generate a snapshot of the state of the instance, and halt the instance. When a request to execute the code is received, the snapshot can be used to quickly restore the instance. The code can then be executed within the instance, reducing the need to initialize the instance or maintain the instance in an executing state.

First claim

What is claimed is:

1. A computer-implemented method comprising:
generating a first software container provisioned with (i) code submitted by a user for execution on an on-demand code execution system and (ii) a runtime on which execution of the code depends, wherein generating the first software container comprises:
selecting a first checkpoint, wherein the first checkpoint reflects a state of a second software container provisioned with the runtime, and wherein the first checkpoint is selected from a plurality of checkpoints based at least in part on a dependency on which execution of the code depends, which dependency is initialized on the second software container whose state is captured within the first checkpoint,
generating the first software container from the first checkpoint of the second software container, and
provisioning the first software container with additional dependency data required to place the runtime into a ready state for execution of the code;

initializing the runtime on the first software container into the ready state for execution of the code;
generating a second checkpoint of the first software container at a time that the runtime on the first software container is initialized into the ready state for execution of the code and prior to execution of the code in the first software container;
storing the second checkpoint as an execution-ready checkpoint for the code;
obtaining a request to execute the code, wherein the request corresponds to the user of the on-demand code execution system; and
in response to the request to execute the code, generating, from the execution-ready checkpoint for the code that reflects the checkpoint of the first software container at a time that the runtime on the first software container is initialized into the ready state for execution of the code and prior to execution of the code in the first software container, a third software container in the ready state for execution of the code, including the initialized runtime, and executing the code within the third software container.

2. The computer-implemented method of claim 1 , further comprising halting execution of the first software container after generating the second checkpoint, wherein generating the third software container from the execution-ready checkpoint for the code comprises generating a new software container and updating the new software container, using the execution-ready checkpoint for the code, to match the ready state.

3. The computer-implemented method of claim 1 , wherein generating the third software container from the execution-ready checkpoint for the code comprises modifying an executing software container, using the execution-ready checkpoint for the code, to match the ready state.

4. The computer-implemented method of claim 1 , wherein generating the first software container provisioned with the runtime comprises provisioning the first software container with additional dependency data required to place the runtime into the ready state for execution of the code.

5. The computer-implemented method of claim 1 , wherein initializing the runtime on the first software container into the ready state for execution of the code comprises executing an initialization portion of the code within the first software container.

6. One or more non-transitory computer-readable media comprising instructions that, when executed by a computing system, cause the computing system to:
generate a first software container provisioned with (i) code submitted by a user for execution on an on-demand code execution system and (ii) a runtime on which execution of the code depends, wherein the instructions cause the computing system to generate the first software container at least partly by:
identifying a dependency on which execution of the code depends,
selecting a first checkpoint storing a state of a software container initialized with the dependency, wherein the first checkpoint is selected from a plurality of checkpoints based at least in part on the dependency on which execution of the code depends, which dependency is initialized on the software container whose state is captured within the first checkpoint, and
generating the first software container based at least in part on the first checkpoint;

initialize the runtime on the first software container into a ready state for execution of the code;
generate a second checkpoint of the first software container at a time that the runtime on the first software container is initialized into the ready state for execution of the code and prior to execution of the code in the first software container;
store the second checkpoint as an execution-ready checkpoint for the code;
obtain a request to execute the code; and
in response to the request to execute the code, generate, from the execution-ready checkpoint, a second software container in the ready state for execution of the code, including the initialized runtime, and execute the code within the second software container.

7. The non-transitory computer-readable media of claim 6 , wherein the instructions cause the computing system to initialize the runtime on the first software container into the ready state for execution of the code based at least partly on executing an initialization portion of the code.

8. The non-transitory computer-readable media of claim 7 , wherein the request to execute the code comprises a parameter to be passed to an execution of the code, and wherein executing an initialization portion of the code places the execution of the code in a state corresponding to a location, within the code, at which the parameter is processed by the execution.

9. The non-transitory computer-readable media of claim 6 , wherein the instructions cause the computing system to generate the second software container from the execution-ready checkpoint for the code at least partly by at least one of generating a new software container and updating the new software container to match the ready state or modifying an executing software container to match the ready state.

10. A system comprising:
a data store including an execution-ready checkpoint for code submitted by a user for execution on an on-demand code execution system, wherein execution of the code depends on a runtime, and wherein the execution-ready checkpoint for the code records a state of a first software container provisioned with the code at a time that the runtime on the first software container is initialized into a ready state for executing the code and prior to execution of the code in the first software container; and
one or more computing devices configured with specific executable instructions to:
generate the execution-ready checkpoint for the code, wherein to generate the execution-ready checkpoint for the code, the one or more computing devices are configured with specific executable instructions to:
identify a dependency on which execution of the code depends,
select a first checkpoint, wherein the first checkpoint stores a state of a container initialized with the dependency, and wherein the first checkpoint is selected from a plurality of checkpoints based at least in part on the dependency on which execution of the code depends, which dependency is initialized on a virtual machine instance whose state is captured within the first checkpoint,
generate the first software container based at least partly on the first checkpoint, and
generate the execution-ready checkpoint for the code from a second checkpoint taken of the first software container;

obtain a request to execute the code, the request corresponding to the user of the on-demand code execution system; and
in response to the request to execute the code, gener

Assignees

• Amazon Tech Inc

Inventors

• Brooker Marc John
• Danilov Mikhail
• Laurence Douglas Stewart
• Liguori Anthony Nicholas

Classifications

• G06F9/45558 Primary

  Hypervisor-specific management and integration aspects · CPC title

• G06F11/1451

  by selection of backup contents · CPC title

• G06F2009/45562

  Creating, deleting, cloning virtual machine instances · CPC title

• G06F2009/45575

  Starting, stopping, suspending or resuming virtual machine instances · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title