Routing child tasks of computational jobs to cells of a network-based service

Patent metadata

Field	Value
Publication number	US-12405963-B1
Application number	US-202418424264-A
Country	US
Kind code	B1
Filing date	Jan 26, 2024
Priority date	Jan 26, 2024
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

Child tasks of computational jobs are routed to cells of a network-based service. A request received at a request router to perform a child task of a computational job may be received. The request router may parse the request to extract a cell identifier for the run of the computational job. The cell identifier may then be used to identify the cell out of multiple cells implementing the network-based service. The cell identifier may be provided by the cell as part of performing a parent task of the run of the computational job. Then, the request may be sent to the identified cell to perform the child task of the run of the computational job.

First claim

What is claimed is:

1. A system, comprising:
a plurality of computing devices, respectively comprising at least one processor and a memory, wherein the plurality of a computing devices implement a job execution service for an Extract Transform and Load (ETL) service offered by a provider network, wherein the job execution service comprises:
a request router;
a plurality of cells as part of a cell-based architecture for the job execution service to execute runs of ETL jobs at the job execution service, wherein the plurality of cells respectively comprise computing resources that implement independent replicas of the job execution service to perform tasks sent by the request router, wherein individual ones of the plurality of cells are isolated from communicating with other ones of the plurality of cells;
wherein the request router is configured to:
receive a request to perform a child task of an ETL job, wherein the ETL job comprises a parent task and one or more child tasks to complete a run of the ETL job;
parse the request to extract a cell identifier for the run of the ETL job included in the request, wherein the cell identifier was previously provided as part performing of the parent task by a cell of the plurality of cells of the job execution service assigned the run of the computational job;
perform a lookup in mapping information between the individual ones of the plurality of cells are to different cell identifiers to identify the cell of the plurality of cells according to the extracted cell identifier; and
send the request to the identified cell to perform the child task of the run of the ETL job.

2. The system of claim 1 , wherein the request router is further configured to:
receive a request to perform the parent task of the run of the computational job;
select the cell to assign the run of the computational job; and
send the request to perform the parent task to the cell.

3. The system of claim 2 , wherein a previously selected cell of the plurality of cells rejected the run of the computational job before the cell was selected to assign the run of the computational job.

4. The system of claim 2 , wherein the request router is further configured to obtain metadata to execute the run of the computational job at the request router, wherein the request to perform the parent task includes at least some of the obtained metadata.

5. A method, comprising:
receiving, at a request router for a network-based service, a request to perform a child task of a computational job, wherein the computational job comprises a parent task and one or more child tasks to complete a run of the computational job;
parsing, by the request router, the request to extract a cell identifier for the run of the computational job included in the request, wherein the network-based service implements a plurality of cells as part of a cell-based architecture for the network-based service to execute runs of computational jobs at the network-based service, wherein the plurality of cells respectively comprise computing resources that implement independent replicas of the network-based service to perform tasks sent by the request router, wherein individual ones of the plurality of cells are isolated from communicating with other ones of the plurality of cells, and wherein the cell identifier was previously provided as part performing of the parent task by a cell of the plurality of cells of the network-based service assigned the run of the computational job;
identifying, by the request router, the cell of the plurality of cells of the network-based service according to a mapping between the cell and the cell identifier, wherein the individual ones of the plurality of cells are mapped to different cell identifiers; and
sending, by the request router, the request to the identified cell to perform the child task of the run of the computational job.

6. The method of claim 5 , further comprising:
receiving, by the request router, a request to perform the parent task of the run of the computational job;
selecting, by the request router, the cell to assign the run of the computational job; and
sending, by the request router, the request to perform the parent task to the cell.

7. The method of claim 6 , wherein the selection of the cell is based on a workload distribution scheme.

8. The method of claim 6 , wherein a previously selected cell of the plurality of cells rejected the run of the computational job before the cell was selected to assign the run of the computational job.

9. The method of claim 6 , wherein the selection is performed according to the cell having been pinned for executing the computational job.

10. The method of claim 6 , further comprising obtaining metadata to execute the run of the computational job at the request router, wherein the request to perform the parent task includes at least some of the obtained metadata.

11. The method of claim 5 , further comprising:
receiving, by the request router, a request to perform a management operation with respect to the run of the computational job, wherein the request specifies the cell identifier of the cell;
parsing, by the request router, the request to perform the management operation to extract the cell identifier included in the request for the information;
identifying, by the request router, the cell of the plurality of cells of the network-based service according to the mapping between the cell and the cell identifier; and
sending, by the request router, the request to perform the management operation to the cell of the network-based service, wherein the cell performs the requested management operation.

12. The method of claim 5 , wherein a different run of the computational job is assigned to a different cell of the plurality of cells of the network-based service.

13. The method of claim 5 , wherein the computational job is an Extract Transform and Load (ETL) job, and wherein the network-based service is an ETL service offered by a provider network.

14. One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement a request router for a network-based service that implements:
receiving a request to perform a child task of a computational job, wherein the computational job comprises a parent task and one or more child tasks to complete a run of the computational job;
parsing the request to extract a cell identifier for the run of the computational job included in the request, wherein the network-based service implements a plurality of cells as part of a cell-based architecture for the network-based service to execute runs of computational jobs at the network-based service, wherein the plurality of cells respectively comprise computing resources that implement independent replicas of the network-based service to perform tasks sent by the request router, wherein individual ones of the plurality of cells are isolated from communicating with other ones of the plurality of cells, and wherein the cell identifier was previously provided as part performing of the parent task by a cell of the plurality of cells of the network-based service assigned the run of the computational job;
performing a lookup in mapping information between the individual ones of the plurality of cells are to different cell identifiers to identify the cell of the plurality of cells according to the extracted cell identifier; and
sending the request to the identified cell to perform the child task of the run of the computational job.

15. The one or

Assignees

• Amazon Tech Inc

Inventors

• Kim Dajung
• Zhang Linchi
• Chandrachood Santosh
• Gawande Pratik Bhagwat
• Bahadur Nitin
• Sitaraman Harish
• Shreekantha Sampath
• Lee Chang Woo

Classifications

• G06F21/6218

  to a system of files or objects, e.g. local or distributed file system or database · CPC title

• G06F16/254 Primary

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title