Data flow management in processing workflows

Patent metadata

Field	Value
Publication number	US-9766927-B1
Application number	US-201514876560-A
Country	US
Kind code	B1
Filing date	Oct 6, 2015
Priority date	Oct 6, 2015
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

A workflow interpreter service that interprets a workflow definition language for specifying a workflow definition. Further, the workflow definition language provides features for maintaining control over data flows for data that is passed from one state to another among states of a state machine for a workflow. Such control over data flow in between states allows for a given workflow to be processed incrementally, and among multiple different computing resources.

First claim

What is claimed is:

1. A system, comprising:
a workflow data store configured to store workflow definitions, wherein respective workflow definitions stored in the data store comprise statements according to a workflow definition language, wherein the statements define states of a respective workflow, and wherein the statements include fields specifying:
one or more tasks to be performed for the respective workflow;
one or more data filters defining a data flow between states of the respective workflow; and
one or more transitions between states of the respective workflow; and

a workflow service implemented by one or more computing devices and configured to:
receive a given workflow definition of the workflow definitions from the data store for a particular workflow;
construct a state machine for the particular workflow as defined in the given workflow definition for the particular workflow; and
execute the particular workflow according to the state machine, wherein to execute the particular workflow, the workflow service is further configured to:
determine transitions between states of the particular workflow as defined in the given workflow definition; and
prior to a particular transition from a current state to a next state of the particular workflow, wherein an input to the current state is an input data flow object:
evaluate the current state to determine one or more tasks to perform for the particular workflow as defined in the given workflow definition for the current state; and
adjust content of an output data flow object to pass between the current state and the next state, wherein the given workflow definition for the current state includes one or more data filters, and wherein the content of the output data flow object is adjusted based at least on one or more of: the input data flow object, an application of the one or more data filters to output from one or more tasks defined for the current state, or an application of the one or more data filters to the input data flow object.

2. The system of claim 1 , wherein the workflow service further comprises:
a workflow handling service implemented by one or more computing devices and configured to:
maintain workflow logs for respective workflows, wherein a particular workflow log of the workflow logs for the particular workflow logs the data flow object for the particular workflow with respect to different events for the particular workflow, wherein different entries in the particular workflow log include different respective versions of the data flow object; and
request a workflow decision for the particular workflow of the respective workflows to determine a task to schedule for the particular workflow.

3. The system of claim 2 , wherein the workflow service further comprises:
a workflow evaluation service implemented by one or more computing devices and configured to:
receive the request for the workflow decision for the particular workflow, wherein the request includes at least a portion of the particular workflow log for the particular workflow including at least a portion of the different versions of the data flow object; and

a workflow definition interpreter implemented by one or more computing devices and configured to interpret the particular workflow definition to:
perform the construct the state machine for the particular workflow as defined in the given workflow definition for the particular workflow;
use the workflow log for the particular workflow to the state machine to transition the state machine to the current state for the particular workflow;
perform the determine transitions between states of the particular workflow;
perform the adjust content of the data flow object from the workflow log; and
perform the evaluate the current state to determine the one or more tasks defined for the current state.

4. The system of claim 3 , wherein
the workflow evaluation service is further configured to:
send the workflow decision for the particular workflow to the workflow handling service, wherein the workflow decision specifies the one or more tasks defined for the current state; and

the workflow handling service is further configured to:
receive the workflow decision from the workflow evaluation service;
log the workflow decision and the adjusted data flow object in the workflow log for the particular workflow; and
schedule for execution the one or more tasks corresponding to the workflow decision.

5. The system of claim 1 , wherein the given workflow definition for the current state of the particular workflow includes a field specifying one of the one or more data filters to be an input data filter that when applied to the data flow object defines filtered data flow input to be used to process the current state.

6. A method, comprising:
performing, by one or more hardware processors:
constructing a state machine for a workflow as defined in a workflow definition for the workflow specified according to a workflow definition language; and
executing the workflow according to the state machine, wherein the executing comprises a workflow definition interpreter performing:
determining transitions between states of the workflow as defined in the workflow definition; and
prior to a particular transition from a current state to a next state of the particular workflow, wherein an input to the current state is an input data flow object:
evaluating the current state to determine one or more tasks to perform for the particular workflow as defined in the given workflow definition for the current state; and
adjusting content of an output data flow object to pass between the current state and the next state wherein the given workflow definition for the current state includes one or more data filters, and wherein the content of the output data flow object is adjusted based at least on one or more of: the input data flow object, an application of the one or more data filters to output from one or more tasks defined for the current state, or an application of the one or more data filters to the input data flow object.

7. The method of claim 6 , wherein the workflow definition for the current state of the workflow includes a field specifying one of the one or more filters to be an input data filter that when applied to the data flow object defines filtered data flow input to be used in the evaluating the current state.

8. The method of claim 6 , wherein the workflow definition for the current state of the workflow includes a field specifying a plurality of state choices, wherein the next state is one of the plurality of state choices, and wherein the method further comprises:
determining the particular transition between the current state and the next state based at least upon the input data flow input to select the next state from among the plurality of state choices.

9. The method of claim 6 , wherein the workflow definition for the current state of the workflow includes a field specifying one of the one or more filters to be an output filter, and includes a field specifying a plurality of state branches, wherein the one or more tasks comprise a plurality of tasks corresponding to the plurality of state branches, and wherein the method further comprises:
invoking each of the plurality of tasks to execute in parallel to generate a plurality of parallel task results, wherein each of the plurality of tasks is invoked using a same input;
wherein the adjusting content of the output data flow object to pass between the current state and the next state is based at least in part on an application of the output filter to the plurality of parallel task results.

10. The method of claim 6 , wherein the data flow object c

Assignees

• Amazon Tech Inc

Inventors

• Islam Zakiul
• Rehaag Aaron-Kenneth Karl
• Bray Timothy William
• Mohan Paritosh
• Yang Yuke
• Clark Jonathan Lewis
• Meng Huangdong
• Schnarr Nathan Andrew
• Ponnau Luc Rémi

Classifications

• G06F17/30424

  Physics · mapped topic

• G06F9/444

  Physics · mapped topic

• G06Q10/06 Primary

  Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling · CPC title

• G06F9/4843 Primary

  by program, e.g. task dispatcher, supervisor, operating system · CPC title

• G06F9/4498

  Finite state machines · CPC title