Job scheduling and monitoring in a distributed computing environment

Patent metadata

Field	Value
Publication number	US-10606665-B2
Application number	US-201715596709-A
Country	US
Kind code	B2
Filing date	May 16, 2017
Priority date	Dec 16, 2014
Publication date	Mar 31, 2020
Grant date	Mar 31, 2020

Abstract

Job execution can be scheduled and monitored. Execution of a job results in generation of one or more data slices, which can be consumed as input by one or more related jobs. Data slices can be presented in an interactive view. Selection of a data slice in the view can trigger automatic identification of related data slices based on dependencies between data slices. Each data slice can also include and be presented with a status indicating the availability of corresponding data, such as pending, ready, or failed. Furthermore, selection of a data slice that failed to generate successfully can trigger identification of related data slices that failed or are predicted to fail to generate successfully.

First claim

What is claimed is:

1. A method performed by a system comprising at least one processor coupled to a memory storing machine-executable instructions, which, when executed by the processor, control the system to perform acts comprising:
conveying, for display on a display device, a timeline view of a data-transformation job schedule in terms of one or more data slices comprising pieces of data of a continuum comprising a data set, wherein each data slice is generated or consumed by a data-transformation job and includes a length, and start and end times;
receiving a first signal from a user selecting a data slice from the one or more data slices in the view;
determining one or more related data slices to the user selected data slice automatically based on data dependencies, wherein the one or more related data slices comprise at least one of one or more data slices that depend from the user selected data slice or one or more data slices from which the user selected data slice depends; and
conveying, for display on the display device, a visualization of the user selected data slice and the one or more related data slices visually distinct from one or more other data slices in the timeline view.

2. The method of claim 1 , wherein the acts further comprise conveying, for display on the display device, one or more data slices scheduled for generation by a job.

3. The method of claim 1 , wherein the acts further comprise conveying, for display on the display device, one or more data slices previously generated by a job.

4. The method of claim 1 , wherein the acts further comprise conveying, for display on the display device, previously generated data slices visually distinct from data slices scheduled for generation by a job.

5. The method of claim 1 , wherein the acts further comprise:
determining compute load during generation of one or more data slices; and
conveying, for display on the display device, the compute load aligned with the one or more data slices.

6. The method of claim 1 , wherein the acts further comprise conveying, for display on the display device, an indication that at least one data sliced failed to generate successfully.

7. The method of claim 6 , wherein the acts further comprise:
receiving a second signal selecting one of at least one data slice that failed to generate successfully;
determining automatically one or more related data slices related to a selected data slice, wherein the related data slices include at least one of one or more data slices from which the selected data slice depends that failed to generate successfully or one or more data slices predicted to fail to generate successfully based on dependency from the selected data slice; and
conveying, for display on the display device, a highlighted version of the selected data slice and the one or more related data slices.

8. The method of claim 6 , wherein the acts further comprise:
receiving a second signal selecting one of the at least one data slice that failed to generate successfully; and
rescheduling generation of the at least one data slice in response to the second signal.

9. The method of claim 1 , wherein the acts further comprise conveying, for display on the display device, the selected data slice and the one or more related data slices in a job authoring diagram comprising representations of jobs and data sets as well as connections between the jobs and data sets.

10. A computer system, comprising:
a processor coupled to a memory, the processor configured to execute machine-executable instructions stored in the memory that when executed perform acts comprising:
conveying, for display on a display device, a timeline view of a data-transformation job schedule in terms of one or more data slices comprising pieces of data of a continuum comprising a data set, wherein each data slice is generated or consumed by a data-transformation job and includes a length, and start and end times;
detecting a first signal from a user selecting a data slice from the one or more data slices in the view;
determining one or more related data slices to the user selected data slice automatically based on data dependencies, wherein the one or more related data slices comprise at least one of one or more data slices that depend from the user selected data slice or one or more data slices from which the user selected data slice depends; and
conveying, for display on the display device, a visualization of the user selected data slice and the one or more related data slices visually distinct from one or more other data slices in the view.

11. The system of claim 10 , wherein the acts further comprise:
monitoring computer resource utilization in conjunction with data slice production; and
conveying, for display on the display device, a visualization of the resource utilization aligned with one or more respective data slices.

12. The system of claim 10 , wherein the acts further comprise:
determining success or failure of data slice generation; and
presenting failed data slice generation different from successful data slice generation.

13. The system of claim 10 , wherein the acts further comprise rescheduling generation of a data slice based on a signal received with respect to a failed data slice generation.

14. The system of claim 10 , wherein the acts further comprise:
detecting a second signal selecting one of at least one data slice where generation failed;
determining automatically one or more related data slices that are related to a selected data slice and predicted to fail based on dependency on the selected data slice; and
conveying, for display on the display device, a highlighted version of the selected data slice and the one or more related data slices.

15. The system of claim 10 , wherein the acts further comprise conveying, for display on the display device, the selected data slice and the one or more related data slices in a job authoring diagram comprising representations of jobs and data sets as well as connections between the jobs and data sets.

16. A computer-readable storage medium having instructions stored thereon that enable at least one processor to perform a method upon execution of the instructions, the method comprising:
conveying, for display on a display device, a timeline view of a data-transformation job schedule in terms of one or more data slices comprising pieces of data of a continuum comprising a data set, wherein each data slice is generated or consumed by a data-transformation job and includes a length, and start and end times;
receiving a first signal from a user selecting a data slice from the one or more data slices in the view;
determining one or more related data slices to the user selected data slice automatically based on data dependencies, wherein the one or more related data slices comprise at least one of one or more data slices that depend from the user selected data slice or one or more data slices from which the user selected data slice depends; and
conveying, for display on the display device, a visualization of the user selected data slice and the one or more related data slices visually distinct from one or more other data slices in the view.

17. The computer-readable storage medium of claim 16 , wherein the method further comprises conveying, for display on the display device, previously produced data slices visually distinct from data slices scheduled for generation.

18. The computer-readable storage medium of claim 16 , wherein the method further comprises:
determining compute load during production of o

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Peacock Andrew J
• Couris Cheryl
• Storm Christina
• Netz Amir
• Cheung Chiu Ying
• Flasko Michael J
• Grealish Kevin
• Della-Libera Giovanni M
• Carlson Sonia P
• Heninger Mark W
• Bach Paula M
• Nettleton David J

Classifications

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/04842 Primary

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06Q10/06311 Primary

  Scheduling, planning or task assignment for a person or group · CPC title

• G06F9/52 Primary

  Program synchronisation; Mutual exclusion, e.g. by means of semaphores · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title