Managing interfaces for sub-graphs

Patent metadata

Field	Value
Publication number	US-10901702-B2
Application number	US-201816200998-A
Country	US
Kind code	B2
Filing date	Nov 27, 2018
Priority date	Dec 5, 2013
Publication date	Jan 26, 2021
Grant date	Jan 26, 2021

Abstract

Combining specifications of dataflow graphs includes receiving: a first dataflow graph specification that specifies two or more components connected by links representing flows of data, and a second dataflow graph specification that specifies at least one component, and at least one sub-graph interface. The sub-graph interface includes at least one flow junction representing a connection between: (1) a flow of data outside the sub-graph interface, and (2) a flow of data inside the sub-graph interface. The method includes processing information including the first dataflow graph specification and the second dataflow graph specification, to generate a combined dataflow graph specification, including: identifying an association between the sub-graph interface and the first dataflow graph specification, for at least a first flow junction, determining a direction associated with transferring a value of a descriptor, and transferring a value of a descriptor according to the determined direction.

First claim

What is claimed is:

1. A method for configuring a dataflow graph template that includes one or more subgraph interfaces, the method including:
accessing one or more subgraph interfaces,
wherein each of the one or more subgraph interfaces is stored,
wherein each of the one or more subgraph interfaces is associated with a unique identifier for providing a reference to the subgraph interface, and
wherein each of the one or more subgraph interfaces has one or more flow junctions;

accessing a dataflow graph template that includes a dataflow graph and that further includes a reference for identifying a given subgraph interface of the one or more subgraph interfaces;
accessing an implementation subgraph that conforms with the given subgraph interface;
configuring the dataflow graph template by:
accessing one or more flow junctions of the given subgraph interface referenced by the dataflow graph template;
linking the implementation subgraph into the dataflow graph template, with the implementation subgraph being linked in accordance with the given subgraph interface in the dataflow graph template; and
determining, for each of the one or more flow junctions of the given subgraph interface, a direction of metadata propagation,
where the determined direction is one of an inward transfer of metadata from the flow junction on the given subgraph interface to a component of the dataflow graph or an outward transfer of the metadata from the flow junction on the given subgraph interface to the component of the dataflow graph; and

storing the configured dataflow graph template.

2. The method of claim 1 wherein the one or more flow junctions define a point of connection between the given sub-graph interface in the dataflow graph template and the implementation subgraph.

3. The method of claim 1 wherein the one or more flow junctions represent corresponding one or more connections between a flow of data to or from a port on a component of the sub-graph interface in the dataflow graph and a flow of data to or from a port on a component of the implementation subgraph.

4. The method of claim 1 , further including:
receiving datasets from a database or from some other data storage or data queueing system;
wherein the dataflow graph includes dataset components representing the received datasets and data processing components, with ports of the dataset components, the data processing components, and the one or more flow junctions of the one or more sub-graph interfaces define data propagation among the dataset components, the data processing components, and the one or more sub-graph interfaces.

5. The method of claim 1 , wherein the dataflow graph template is specified as read-only, encrypted or protected from being viewed and/or altered.

6. The method of claim 1 , wherein the implementation subgraph is specified as read-only, encrypted or protected from being viewed and/or altered.

7. The method of claim 1 , wherein the dataflow graph template is configured to accommodate different types of record formats from the implementation subgraph without requiring user intervention.

8. The method of claim 1 , wherein a flow junction is an output flow function or an input flow junction, with the method further including:
adding an input flow junction or an output flow junction that are represented as one or more menu items from a click menu in a graphical user interface.

9. The method of claim 1 , wherein a flow junction is an output flow function or an input flow junction, with the method further including:
removing an input flow junction or an output flow junction that is represented as one or more menu items from a click menu in a graphical user interface.

10. The method of claim 1 , wherein a flow junction is an output flow function or an input flow junction, with the method further including:
adding an input and/or an output flow junction to the one or more sub-graph interfaces.

11. The method of claim 1 , wherein a flow junction is an output flow function or an input flow junction, with the method further including:
generating one or more links between input flow junctions and output flow junctions, with a generated link between a given one of the input flow junctions and a given one of the output flow junctions indicative of relationships between metadata associated with the given one of the input flow junctions and metadata associated with the given one of the output flow junctions.

12. The method of claim 1 , further including generating a generated link through a user interface.

13. The method of claim 1 further including:
generating a graphical user interface to render a flow junction configuration tab to configure the one or more flow junctions of the one or more sub-graph interfaces, with the flow junction configuration tab including a flow junction list and a propagation control.

14. The method of claim 13 wherein the flow junction list renders a list of flow junctions and the propagation control includes one or more controls to select the direction of metadata propagation as an inward transfer direction or an outward transfer direction.

15. The method of claim 1 , further including analyzing at least one of the one or more flow junctions of the one or more subgraph interfaces to determine the direction of metadata propagation.

16. The method of claim 1 , wherein accessing the one or more subgraph interfaces includes:
accessing one or more subgraph interfaces that each includes a flow junction representing a connection between (i) a flow of data outside the subgraph interface to or from a component of the dataflow graph, and (ii) a flow of data inside the subgraph interface to or from a component of the dataflow graph,
wherein the implementation subgraph is configured to be linked into the dataflow graph template by:
accessing one or more flow junctions of the given subgraph interface to determine a direction of metadata propagation between a component of the implementation subgraph and a component of the dataflow graph,

wherein each flow junction of the given subgraph interface specifies a direction of metadata propagation; and
wherein determining includes:
determining, for each of the one or more accessed flow junctions of the given subgraph interface, a direction of metadata propagation between the component of the implementation subgraph that conforms with the subgraph interface and the component of the dataflow graph,
where the determined direction is one of an inward transfer of metadata into the flow junction of the given subgraph interface from the component of the dataflow graph, or an outward transfer of the metadata from the flow junction of the given subgraph interface to the component of the dataflow graph.

17. The method of claim 1 , wherein linking and determining each occur prior to execution of the dataflow graph with the linked implementation subgraph.

18. The method of claim 1 , further including:
independent of characteristics of a flow of data, executing the dataflow graph with the linked implementation subgraph.

19. The method of claim 1 , including:
prior to executing the dataflow graph, performing link-time metadata resolution that resolves a placeholder value at a flow junction in the implementation sub-graph by replacing the placeholder value with a specified metadata value.

20. The method of claim 1 , wherein each of the one or more subgraph interfaces is associated with a function; and
wherein dynamically linking the implementation subgraph in place of the given

Assignees

• Ab Initio Technology Llc

Inventors

• Larson Brond
• Bay Paul
• Bromley H Mark

Classifications

• G06F16/9024

  Graphs; Linked lists (G06F16/9027 takes precedence) · CPC title

• G06F8/10

  Requirements analysis; Specification techniques · CPC title

• G06F8/34 Primary

  Graphical or visual programming · CPC title

• G06F8/20

  Software design · CPC title

• G06F9/4494

  data driven · CPC title