Parameter variations for computations using a remote repository

Patent metadata

Field	Value
Publication number	US-10983830-B2
Application number	US-201816147474-A
Country	US
Kind code	B2
Filing date	Sep 28, 2018
Priority date	Sep 28, 2018
Publication date	Apr 20, 2021
Grant date	Apr 20, 2021

Abstract

Methods, systems, and computer-readable media for parameter variations for computations using a remote repository are disclosed. A first computation is assigned to a first set of one or more hosts. The first computation is associated with first parameters including one or more inputs and program code. A second computation is assigned to a second set of one or more hosts and is associated with a second set of parameters. Execution of the first computation is initiated using the first set of hosts and the first set of parameters. Local copies of the input(s) and program code are obtained from a storage service using a TO credential supplied by a repository manager. Execution of the second computation is initiated using the second set of hosts and the second set of parameters as obtained using a credential supplied by the repository manager.

First claim

What is claimed is:

1. A system, comprising:
a repository manager;
a plurality of hosts; and
a computation orchestrator implemented using one or more computing devices, wherein the computation orchestrator is configured to:
assign a first computation to a first set of one or more hosts of the plurality of hosts, wherein the first computation is associated with a first set of parameters including a first set of one or more inputs and a first set of program code;
assign a second computation to a second set of one or more hosts of the plurality of hosts, wherein the second set of one or more hosts is different than the first set of one or more hosts, and wherein the second computation is associated with a second set of parameters that either comprises the first set of program code and a second set of one or more inputs that differs at least in part from the first set of one or more inputs, or comprises the first set of one or more inputs and a second set of program code that differs at least in part from the first set of program code;
initiate execution of the first computation using the first set of one or more hosts and using the first set of parameters, wherein a local copy of the first set of one or more inputs and a local copy of the first set of program code are obtained by the first set of one or more hosts from a storage service using one or more credentials supplied by the repository manager, and wherein the first computation is executed by the first set of one or more hosts using the local copy of the first set of program code and the local copy of the first set of one or more inputs;
initiate execution of the second computation using the second set of one or more hosts and using the second set of parameters;
store, using the repository manager, a representation of one or more outputs of the first computation and a representation of one or more outputs of the second computation.

2. The system as recited in claim 1 , wherein the second computation is executed by the second set of one or more hosts using an additional local copy of the first set of program code and a local copy of the second set of one or more inputs, or wherein the second computation is executed by the second set of one or more hosts using a local copy of the second set of program code and an additional local copy of the first set of one or more inputs.

3. The system as recited in claim 1 , wherein the first computation and the second computation are run in parallel, and wherein the computation orchestrator is configured to:
perform a comparison of the outcome of the first computation to an outcome of the second computation; and
select the first set of parameters or the second set of parameters responsive to the comparison.

4. The system as recited in claim 1 , wherein the execution of the second computation is initiated after the execution of the first computation is complete, and wherein the computation orchestrator is configured to:
vary the second set of parameters from the first set of parameters responsive to an outcome of the first computation.

5. A computer-implemented method performed by one or more computing devices, comprising:
assigning a first computation to a first set of one or more hosts, wherein the first computation is associated with a first set of parameters including a first set of one or more inputs and a first set of program code;
assigning a second computation to a second set of one or more hosts, wherein the second set of one or more hosts is different than the first set of one or more hosts, and wherein the second computation is associated with a second set of parameters that either comprises the first set of program code and a second set of one or more inputs that differs at least in part from the first set of one or more inputs, or comprises the first set of one or more inputs and a second set of program code that differs at least in part from the first set of program code;
initiating execution of the first computation using the first set of one or more hosts and using the first set of parameters, wherein a local copy of the first set of one or more inputs and a local copy of the first set of program code are obtained by the first set of one or more hosts from a storage service using one or more credentials supplied by a repository manager, and wherein the first computation is executed by the first set of one or more hosts using the local copy of the first set of program code and the local copy of the first set of one or more inputs; and
initiating execution of the second computation using the second set of one or more hosts and using the second set of parameters.

6. The method as recited in claim 5 , wherein the second computation is executed by the second set of one or more hosts using an additional local copy of the first set of program code and a local copy of the second set of one or more inputs, wherein the additional local copy of the first set of program code and the local copy of the second set of one or more inputs are obtained from the repository manager, and wherein the local copy of the second set of one or more inputs is accessible by the second computation via a local filesystem mounted at the second set of one or more hosts.

7. The method as recited in claim 5 , wherein the second computation is executed by the second set of one or more hosts using a local copy of the second set of program code and an additional local copy of the first set of one or more inputs, wherein the local copy of the second set of program code and the additional local copy of the first set of one or more inputs are obtained from the repository manager, and wherein the additional local copy of the first set of one or more inputs is accessible by the second computation via a local filesystem mounted at the second set of one or more hosts.

8. The method as recited in claim 5 , further comprising:
storing, using the repository manager, a representation of one or more outputs of the first computation and a representation of one or more outputs of the second computation.

9. The method as recited in claim 5 , wherein the first computation and the second computation are run in parallel, and wherein the method further comprises:
performing a comparison of the outcome of the first computation to an outcome of the second computation; and
selecting the first set of parameters or the second set of parameters responsive to the comparison.

10. The method as recited in claim 5 , wherein the execution of the second computation is initiated after the execution of the first computation is complete, and wherein the method further comprises:
varying the second set of parameters from the first set of parameters responsive to an outcome of the first computation.

11. The method as recited in claim 5 , wherein the first computation and the second computation include one or more simulations.

12. The method as recited in claim 5 , wherein the first computation and the second computation include one or more machine learning tasks.

13. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more processors, perform:
assigning a first computation to a first set of one or more hosts, wherein the first computation is associated with a first set of parameters including a first set of one or more inputs and a first set of program code;
assigning a second computation to a second set of one or more hosts, wherein the second set of one or more hosts is different than the first set of one or more hosts, and wherein the second computation is associated with a second set of parameters that either comprises the first set

Assignees

• Amazon Tech Inc

Inventors

• Theimer Marvin Michael
• Ellie Julien Jacques
• Watson Colin
• Sankhla Ullas
• Singh Swapandeep
• Hart Kerry
• Anderson Paul
• Dahmen Brian
• Nandini Suchi
• Chen Yunhan
• Liu Shu
• Raman Arjun
• Xie Yuxin
• Xiong Fengjia

Classifications

• G06F9/5027 Primary

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F8/60

  Software deployment · CPC title