Managing job status

Patent metadata

Field	Value
Publication number	US-9886473-B2
Application number	US-201414539072-A
Country	US
Kind code	B2
Filing date	Nov 12, 2014
Priority date	Nov 12, 2013
Publication date	Feb 6, 2018
Grant date	Feb 6, 2018

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for job management. One of the methods includes receiving a status report for each of a plurality of jobs from a pluggable status reporter. The jobs are executed by applications in a distributed computing system. The applications are developed in at least two different software frameworks. A job type for each job is determined. A specific status handler associated with the job type is selected to determine specified status information for the job. The specified status information is stored and associated with the job in a status database.

First claim

What is claimed is:

1. A computer-implemented method comprising:
executing, by a plurality of computers of a distributed computing system, a plurality of respective job status reporter applications, each respective job status reporter application being configured to generate job status reports specific to jobs performed by applications developed in a respective software framework,
wherein the jobs include at least a first job executed by a first computer and developed in a first software framework and a second job executed by a second computer and developed in a second software framework different from the first software framework;
generating, by a first job status reporter application specific to a first software framework, a first job status report for a first application executed by a first computer of the plurality of computers, the first application being a software application developed in the first software framework;
generating, by a second job status reporter application specific to a different second software framework, a second job status report for a second application executed by a second computer of the plurality of computers, the second application being a software application developed in the second software framework;
receiving, by a status monitor application executed by one or more of the plurality of computers in the distributed computing system, the first status report generated by the first job status reporter application and the second status report generated by the second job status reporter application;
determining, by the status monitor application, a first job type for the first job based on the first status report;
determining, by the status monitor application, a second job type for the second job based on the second status report;
determining, by the status monitor application for each of the first job and the second job, a respective status handler application specific to the respective job type from a first database that maps a plurality of job types to respective status handler applications, including identifying a first status handler application specific to a first job type of the first job and a different second status handler application specific to a second job type of the second job;
processing, by the first status handler application, the first status report generated by the first job status reporter application specific to the first software framework of the first job to generate first job status information using only a portion of data from the first status report;
processing, by the second status handler application, the second status report generated by the second job status reporter application specific to the second software framework of the second job to generate second job status information using only a portion of data from the second status report;
storing, by the first status handler application, the generated first job status information in association with the first job in a second database that maps a plurality of jobs to respective job status information; and
storing, by the second status handler application, the generated second job status information in association with the second job in the second database.

2. The method of claim 1 ,
wherein the first job status reporter application and the second job status reporter application are instances of a same job status reporter application.

3. The method of claim 1 , wherein the first status report corresponds to a first step of the first job, the first job comprising a plurality of steps to be executed in the distributed computing system.

4. The method of claim 3 , wherein the first status report for the first step is generated at a generation time point, the generation time point being at a beginning of execution of the first step, at a first time point during the execution of the first step, at an end of the execution of the first step, or at a second time point after the execution of the first step.

5. The method of claim 3 , further comprising determining, by the status monitor application, that the first status report of the first step is received, and in response, transmitting, by the status monitor application, a trigger signal instructing the first computer to continue executing the first job in the distributed computing system.

6. The method of claim 3 , wherein the plurality of steps of the particular job are executed based on a pre-determined schedule for the particular job.

7. The method of claim 6 , wherein the pre-determined schedule is scheduled by a job scheduler, the job scheduler scheduling the plurality of steps of the particular job to be executed in the distributed computing system.

8. The method of claim 3 , wherein each step of the plurality of steps is scheduled to be executed in a respective computer of the plurality of computers in the distributed computing system, and further comprising receiving one or more status reports for the step from a respective status reporter in the respective computer.

9. The method of claim 3 , further comprising:
determining a respective step type for each of the plurality of steps; and
determining, for each of the steps, that the step type for the step is stored in a third database, the third database associating step types to respective step status handler applications, and in response:
selecting a respective step status handler application associated with the step type from the third database;
determining, with the respective step status handler application, specified step status information of the step based on the one or more status reports of the step;
storing the specified step status information in the second database; and
associating the specified step status information with the step of the first job in the second database.

10. The method of claim 1 , wherein processing, by the first status handler application, the first status report generated by the first job status reporter application specific to the first software framework of the first job to generate first job status information comprises generating the first job status information of the first job based on one or more rules that are pre-configured for the first status handler application.

11. The method of claim 1 , further comprising:
receiving a request to query a status of the first job;
retrieving, based on the request, relevant job status information from the second database; and
providing the first job status information for the first job in response to receiving the request.

12. A distributed computing system comprising:
a plurality of computers; and
one or more computer-readable storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
executing, by a plurality of computers of the distributed computing system, a plurality of respective job status reporter applications, each respective job status reporter application being configured to generate job status reports specific to jobs performed by applications developed in a respective software framework,
wherein the jobs include at least a first job executed by a first computer and developed in a first software framework and a second job executed by a second computer and developed in a second software framework different from the first software framework;
generating, by a first job status reporter application specific to a first software framework, a first job status report for a first application executed by a first computer of the plurality of computers, the first application being a software application developed in the first software framework;
generating,

Assignees

• Pivotal Software Inc

Inventors

• Zhao Yanhui
• Gao Xiaoming
• Han Leng
• Li Xiong
• Li Ye

Classifications

• G06F11/3495 Primary

  for systems · CPC title

• G06F11/3065

  Monitoring arrangements determined by the means or processing involved in reporting the monitored data (error or fault reporting or logging G06F11/0766) · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• G06F16/28

  Databases characterised by their database models, e.g. relational or object models · CPC title

• G06F16/21

  Design, administration or maintenance of databases · CPC title