Workflow optimization

Patent metadata

Field	Value
Publication number	US-11315014-B2
Application number	US-201815998728-A
Country	US
Kind code	B2
Filing date	Aug 16, 2018
Priority date	Aug 16, 2018
Publication date	Apr 26, 2022
Grant date	Apr 26, 2022

Abstract

A computer implemented method, computer program product, and system for managing execution of a workflow comprising a set of subworkflows, comprising optimizing the set of subworkflows using a deep neural network, wherein each subworkflow of the set of subworkflows has a set of tasks, wherein each task of the sets of tasks has a requirement of resources of a set of resources; wherein each task of the sets of tasks is enabled to be dependent on another task of the sets of tasks, training the deep neural network by: executing the set of subworkflows, collecting provenance data from the execution, and collecting monitoring data that represents the state of said set of resources, wherein the training causes the neural network to learn relationships between the states of said set of resources, the said sets of tasks, their parameters and the obtained performance, optimizing an allocation of resources of the set of resources to each task of the sets of tasks to ensure compliance with a user-defined quality metric based on the deep neural network output.

First claim

What is claimed is:

1. A computer implemented method for managing execution of a workflow comprising a set of subworkflows, the method comprising:
optimizing the set of subworkflows using a deep neural network; wherein each subworkflow of the set of subworkflows has a set of tasks; wherein each task of the sets of tasks has a requirement of resources of a set of resources; wherein each task of the sets of tasks is enabled to be dependent on another task of the sets of tasks;
training the deep neural network by: executing the set of subworkflows, collecting provenance data from the execution, and collecting monitoring data that represents a state of the set of resources; wherein the training causes the neural network to learn relationships between the states of the set of resources, the sets of tasks, their parameters and an obtained performance, and wherein the relationships between the states of the set of resources, the sets of tasks, their parameters, and the obtained performance are non-linear; and
optimizing an allocation of resources of the set of resources to each task of the sets of tasks to ensure compliance with a user-defined quality metric based on output of the deep neural network, wherein the optimization further comprises creating at least one non-linear model based on the relationships between the states of the set of resources, the sets of tasks, their parameters, and the obtained performance.

2. The method of claim 1 further comprising:
using the optimized allocation of resources to encode a set of chromosomes;
using a Genetic Algorithm, running the set of chromosomes through the deep neural network to determine a new allocation of resources;
determining if the new allocation of resources better complies with a quality of services; and
implementing the new allocation of resources if it does better comply with the quality of services.

3. The method of claim 2 wherein the set of resources includes hardware resources and wherein an amount of hardware resources is enabled to change over execution of the workflows.

4. The method of claim 2 wherein the workflow is represented by a directed acyclic graph.

5. The method of claim 4 wherein inputs for the deep neural network are translated into a tensor representation.

6. A computer program product for managing execution of a workflow comprising a set of subworkflows, the computer program product comprising:
a non-transitory computer readable medium encoded with computer executable program code, the code configured to enable the execution of:
optimizing the set of subworkflows using a deep neural network; wherein each subworkflow of the set of subworkflows has a set of tasks; wherein each task of the sets of tasks has a requirement of resources of a set of resources; wherein each task of the sets of tasks is enabled to be dependent on another task of the sets of tasks;
training the deep neural network by: executing the set of subworkflows, collecting provenance data from the execution, and collecting monitoring data that represents a state of the set of resources; wherein the training causes the neural network to learn relationships between the states of the set of resources, the sets of tasks, their parameters and an obtained performance, and wherein the relationships between the states of the set of resources, the sets of tasks, their parameters, and the obtained performance are non-linear; and
optimizing an allocation of resources of the set of resources to each task of the sets of tasks to ensure compliance with a user-defined quality metric based on output of the deep neural network, wherein the optimization further comprises creating at least one non-linear model based on the relationships between the states of the set of resources, the sets of tasks, their parameters, and the obtained performance.

7. The computer program product of claim 6 the code further configured for:
using the optimized allocation of resources to encode a set of chromosomes;
using a Genetic Algorithm, running the set of chromosomes through the deep neural network to determine a new allocation of resources;
determining if the new allocation of resources better complies with a quality of services; and
implementing the new allocation of resources if it does better comply with the quality of services.

8. The computer program product of claim 7 wherein a set of resources includes hardware resources and wherein an amount of hardware resources is enabled to change over execution of the workflows.

9. The computer program product of claim 7 wherein the workflow is represented by a directed acyclic graph.

10. The computer program product of claim 9 wherein inputs for the deep neural network are translated into a tensor representation.

11. A system for managing execution of a workflow comprising a set of subworkflows, the system comprising:
one or more processors; and
computer executable program code, the code configured to enable the execution across the one or more processors of:
optimizing the set of subworkflows using a deep neural network; wherein each subworkflow of the set of subworkflows has a set of tasks; wherein each task of the sets of tasks has a requirement of resources of a set of resources; wherein each task of the sets of tasks is enabled to be dependent on another task of the sets of tasks;
training the deep neural network by: executing the set of subworkflows, collecting provenance data from the execution, and collecting monitoring data that represents a state of the set of resources; wherein the training causes the neural network to learn relationships between the states of the set of resources, the sets of tasks, their parameters and an obtained performance, and wherein the relationships between the states of the set of resources, the sets of tasks, their parameters, and the obtained performance are non-linear; and
optimizing an allocation of resources of the set of resources to each task of the sets of tasks to ensure compliance with a user-defined quality metric based on output of the deep neural network, wherein the optimization further comprises creating at least one non-linear model based on the relationships between the states of the set of resources, the sets of tasks, their parameters, and the obtained performance.

12. The system of claim 11 the computer executable program code further enabling the execution of:
using the optimized allocation of resources to encode a set of chromosomes;
using a Genetic Algorithm, running the set of chromosomes through the deep neural network to determine a new allocation of resources;
determining if the new allocation of resources better complies with the a quality of services; and
implementing the new allocation of resources if it does better comply with the quality of services.

13. The system of claim 12 wherein the set of resources includes hardware resources and wherein the amount of hardware resources is enabled to change over execution of the workflows.

14. The system of claim 12 wherein the workflow is represented by a directed acyclic graph.

15. The system of claim 14 wherein inputs for the deep neural network are translated into a tensor representation.

16. A computer implemented method, comprising:
optimizing a workflow hierarchy using a deep neural network, wherein the workflow hierarchy comprises at least one workflow having a corresponding set of subworkflows; wherein each subworkflow of the set of subworkflows has a set of tasks; wherein each task of the sets of tasks has a requirement of resources of a set of resources; wherein each task of the sets of tasks is

Assignees

• Emc Ip Holding Co Llc

Inventors

• Dias Jonas F
• Ciarlini Angelo
• Pinho Romulo D
• Gottin Vinicius
• Maximo Andre
• Pacheco Edward
• Holmes David
• Rangarajan Keshava
• Senften Scott David
• Winston Joseph Blake
• Wang Xi
• Walker Clifton Brent
• Dev Ashwani
• Yeleshwarapu Chandra
• Srinivasan Nagaraj

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F9/5038 Primary

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06N3/126 Primary

  Evolutionary algorithms, e.g. genetic algorithms or genetic programming · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title