Automated invalidation of job output data in a job-processing system

Patent metadata

Field	Value
Publication number	US-9465653-B2
Application number	US-201314103683-A
Country	US
Kind code	B2
Filing date	Dec 11, 2013
Priority date	Dec 11, 2013
Publication date	Oct 11, 2016
Grant date	Oct 11, 2016

Abstract

A computing system can provide user interfaces and back-end operations to facilitate review and invalidation of executed jobs. The system can provide an interface that allows the operator to review quality-control information about a completed job. Once the operator identifies a job as invalid, the operator can be presented with further options, such as whether to invalidate only the reviewed job or the job and all its descendants. The operator can also review antecedent jobs to an invalid job (e.g., in order to trace the root of the problem) and can selectively invalidate antecedent jobs.

First claim

What is claimed is:

1. A method comprising:
at one or more computing devices comprising one or more processors and one or more storage media storing one or more computer programs executed by the one or more processors to perform the method, performing operations comprising:
receiving, via a graphical user interface, an input identifying a base job to be invalidated, the base job having been executed to completion or failure, wherein execution of the base job produced at least one output data object;
in response to receiving the input, identifying a set of antecedent jobs of the base job, each of the antecedent jobs in the identified set of antecedent jobs having been executed to completion or failure, wherein each of the antecedent jobs in the set produced at least one output data object that was consumed directly or indirectly by the base job;
identifying a subset of the set of antecedent jobs for invalidation;
invalidating the base job and the subset of the antecedent jobs;
wherein invalidating the base job comprises storing, for the base job, job execution indicia specifying that the at least one output data object produced by the base job is invalid; and
wherein invalidating the subset of the antecedent jobs comprises storing, for each antecedent job in the subset, job execution indicia specifying that the at least one output data object produced by the each antecedent job is invalid.

2. The method of claim 1 further comprising:
presenting, by a monitoring interface, quality-control information about the base job to the user,
wherein receiving the input includes receiving a user input responsive to presenting the quality-control information about the base job.

3. The method of claim 1 wherein identifying the set of antecedent jobs for the base job includes:
identifying a first source data object that is consumed directly by the base job; and
identifying, as one of the antecedent jobs, a first source job that produced the first source data object.

4. The method of claim 3 wherein identifying the set of antecedent jobs for the base job further includes:
identifying a second source data object that is consumed directly by the first source job; and
identifying, as another one of the antecedent jobs, a second source job that produced the second source data object.

5. The method of claim 1 wherein the set of antecedent jobs includes every job executed to completion or failure that produced a data object that was consumed directly or indirectly by the base job.

6. The method of claim 1 wherein identifying the subset of antecedent jobs for invalidation includes:
presenting, by a monitoring interface, quality-control information about one of the antecedent jobs from the set of antecedent jobs to the user; and
receiving, in response to the presentation, a user input indicating whether the presented antecedent job should be invalidated.

7. A computer system comprising:
one or more processing units; and
one or more storage media storing instructions for execution by the one or more processing units and configured for:
identifying a base job to be invalidated, the base job having been executed to completion or failure, wherein execution of the base job produced at least one output data object;
identifying a set of antecedent jobs, each of the antecedent jobs in the identified set of antecedents jobs having been executed to completion or failure, wherein each of the antecedent jobs in the set of antecedent jobs produced at least one output data object that was consumed directly or indirectly by the base job;
identifying a subset of the set of antecedent jobs to be invalidated;
identifying a set of descendant jobs to be invalidated, each of the descendant jobs in the identified set of descendant jobs having been executed to completion or failure, wherein each job in the set of descendant jobs directly or indirectly consumed the at least one output data object that was produced by the base job, wherein each job in the set of descendant jobs produced at least one output data object based on the at least one output data object produced by the base job that the each descendant job directly or indirectly consumed;
invalidating the base job by storing job execution indicia specifying that the at least one output data object produced by the base job is invalid; and
invalidating one or more additional jobs selected from the subset of the antecedent jobs and the set of descendant jobs by storing, for each additional job of the one or more additional jobs, job execution indicia specifying that the at least one output data object produced by the each additional job is invalid.

8. The computer system of claim 7 wherein the instructions further comprise instructions for:
presenting at a user interface, prior to invalidating the base job and the one or more additional jobs, a list of proposed jobs to invalidate, the list including the base job, the subset of the antecedent jobs, and the set of descendant jobs; and
receiving, via the user interface, an instruction indicating which of the jobs on the list of proposed jobs to invalidate should be invalidated.

9. The computer system of claim 7 wherein the instructions further comprise instructions for:
determining whether the one or more invalidated jobs should be rescheduled; and
rescheduling the invalidated jobs in response to determining that the invalidated jobs should be rescheduled.

10. The computer system of claim 9 wherein the instructions further comprise instructions for determining whether the one or more invalidated jobs should be rescheduled based on user input.

11. The computer system of claim 7 wherein invalidating the one or more additional jobs includes invalidating all of the jobs in the set of descendant jobs.

12. The computer system of claim 7 wherein invalidating the one or more additional jobs includes invalidating all of the jobs in the subset of the antecedent jobs.

13. The computer system of claim 7 wherein the instructions further comprise instructions for:
analyzing quality-control data for a plurality of executed jobs to identify one or more suspect jobs whose output data is suspect according to a quality-control metric;
presenting, at a user interface, a listing of the suspect jobs;
receiving, via the user interface, a first user input selecting one of the suspect jobs;
presenting, at the user interface, a representation of the quality-control data for the selected suspect job; and
receiving, via the user interface, a second user input indicating whether the selected suspect job should be invalidated,
wherein the selected suspect job is identified as the base job in response to the second user input.

14. The computer system of claim 7 wherein the instructions further comprise instructions for:
identifying an import job that consumed data from an external source;
determining a delivery time at which the data from the external source was delivered;
determining an execution time at which the import job was executed; and
in the event that the delivery time is later than the execution time:
invalidating the import job; and
invalidating one or more descendant jobs of the import job, wherein each descendant job of the import job directly or indirectly consumed an output data object that was produced by the import job.

15. A non-transitory computer-readable medium having stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to execute a method comprising:
identifying a base job to be invalidated, the base job having been execut

Assignees

• Dropbox Inc

Inventors

• Kishore Shaunak
• Dray Karl

Classifications

• G06F11/3065

  Monitoring arrangements determined by the means or processing involved in reporting the monitored data (error or fault reporting or logging G06F11/0766) · CPC title

• G06F9/4843 Primary

  by program, e.g. task dispatcher, supervisor, operating system · CPC title

• G06F17/40

  Data acquisition and logging (for input to computer G06F3/00) · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title