Data flow design with static and dynamic elements

Patent metadata

Field	Value
Publication number	US-2017329786-A1
Application number	US-201615152502-A
Country	US
Kind code	A1
Filing date	May 11, 2016
Priority date	May 11, 2016
Publication date	Nov 16, 2017
Grant date	—

Abstract

A data flow design system is presented that includes support for both static elements and dynamic elements. Thus, the data flow design system provides a design environment graphical tool to users to design data flows that leverage both the repeatability of static elements and the adaptability of dynamic elements. Static elements process data that typically do not change over time, while dynamic elements process data that do change. For instance, new data fields are added to an input data source of a data flow. The dynamic elements automatically link the new data fields to mapping fragments of the data flow. Mapping fragments process data based on configuration parameters including expression and filter rules. Users use a user interface of the design environment to view and add mapping fragments, static links, and dynamic links to the data flow.

First claim

What is claimed is:

1 . A method comprising:
receiving, in a data integration development environment, a definition of a data flow modeling a data mapping, comprising:
receiving a plurality of mapping fragments for inclusion in the data flow, wherein each mapping fragment comprises a plurality of ports, the plurality of ports including at least one dynamic port or one static port, a dynamic port corresponding to any number of ports of the plurality of ports, a static port corresponding to one port of the plurality of ports, each port corresponding to at least one data field;
receiving input creating at least one dynamic link between a dynamic port of an upstream mapping fragment of the plurality of mapping fragments and a dynamic port of a downstream mapping fragment of the plurality of mapping fragments, the dynamic link providing all data fields of the dynamic port to the dynamic port;
receiving input creating at least one static link between a static port of an upstream mapping fragment of the plurality of mapping fragments and a static port of a downstream mapping fragment of the plurality of mapping fragments, the static link providing all data fields of the static port to the static port;
receiving one or more configuration parameters to apply to at least one mapping fragment of the plurality of mapping fragments, each configuration parameter including at least one configuration parameter value; applying the one or more configuration parameters to the at least one mapping fragment by replacing each configuration parameter value of the one or more configuration parameters with a corresponding runtime value; and compiling an executable runtime definition based at least in part on the definition of the data flow and the runtime values.

2 . The method of claim 1 , further comprising:
executing the executable runtime definition; and generating a report based at least in part on the executing and input data from a database.

3 . The method of claim 2 , wherein receiving the definition of the data flow further comprises:
receiving a runtime link between a reference upstream mapping fragment of the plurality of mapping fragment and a reference downstream mapping fragment of the plurality of mapping fragment; and wherein executing the executable runtime definition comprises resolving the runtime link by providing all data fields of one or more ports of the reference upstream mapping fragment to one or more ports of the reference downstream mapping fragment.

4 . The method of claim 2 , wherein each mapping fragment of the plurality of mapping fragment corresponds to an input data source, output data source, or transformation.

5 . The method of claim 4 , further comprising:
modifying a mapping fragment of the plurality of mapping fragment corresponding to an input data source or an output data source; and wherein executing the executable runtime definition does not require re-compiling the executable runtime definition.

6 . The method of claim 1 , wherein at least one configuration parameter of the one or more configuration parameters is a rule describing an exclusionary or inclusionary filter, and wherein applying the one or more configuration parameters to the at least one mapping fragment further comprises:
replacing the configuration parameter values of the at least one configuration parameter with a corresponding runtime value based on a type of data field indicated by the exclusionary filter.

7 . The method of claim 1 , wherein at least one configuration parameter of the one or more configuration parameters is a proxy indicating a reference dynamic port of a mapping fragment of the plurality of mapping fragment, and wherein applying the one or more configuration parameters to the at least one mapping fragment further comprises:
replacing the configuration parameter values of the at least one configuration parameter value with a corresponding runtime value based on the reference dynamic port.

8 . The method of claim 1 , wherein at least one configuration parameter of the one or more configuration parameters is a macro corresponding to a transformation, and wherein applying the one or more configuration parameters to the at least one mapping fragment further comprises:
replacing the configuration parameter values of the at least one configuration parameter with a corresponding runtime value by applying the transformation over each port of the plurality of ports of the at least one mapping fragment.

9 . The method of claim 1 , wherein receiving the input creating the at least one dynamic link between the dynamic port of the upstream mapping fragment of the plurality of mapping fragment and the dynamic port of the downstream mapping fragment of the plurality of mapping fragment, further comprises generating a plurality of generated ports, corresponding to all data fields of the dynamic port, for the dynamic port of the downstream mapping fragment.

10 . A non-transitory computer-readable memory storing a computer program executable by a processor, the computer program producing a user interface, the user interface comprising:
a data flow display area for displaying a data flow; a data flow icon selection area adjacent to the data flow display area comprising a plurality of icons for adding mapping fragments to the data flow; and wherein the user interface is configured by the computer program to:
in response to receiving input selecting an icon of the plurality of icons in the data flow icon selection, display a new mapping fragment in the data flow display area, the mapping fragments including at least one dynamic port or one static port, where a dynamic port corresponds to a group of two or more data fields, and a static port corresponds to one data field;
in response to receiving input dragging a group of two or more data fields of an upstream mapping fragment in the data flow display area to a downstream mapping fragment in the data flow display area:
displaying a visual connection between the upstream mapping fragment and the downstream mapping fragment indicative of a dynamic link; and
adding the dragged group of two or more fields of the dynamic port of the upstream mapping fragment to a newly generated port in the downstream mapping fragment.

11 . The computer program of claim 10 , wherein the user interface is further configured by the computer program to:
in response to receiving input dragging a data field of an upstream mapping fragment in the data flow display area to a downstream mapping fragment in the data flow display area:
displaying a visual connection between the upstream mapping fragment and the downstream mapping fragment indicative of a static link; and
adding the dragged data field of the static port of the upstream mapping fragment to a newly generated port in the downstream mapping fragment.

12 . The computer program of claim 10 , wherein the user interface further comprises a runtime visual connection between an upstream mapping fragment of the data flow and a downstream mapping fragment of the data flow indicative of a runtime linking, the runtime visual connection visually different than the visual connection indicative of a dynamic linking.

13 . A computer program product comprising a non-transitory computer readable storage medium having instructions encoded therein that, when executed by a processor, cause the processor to:
receive, in a data integration development environment, a definition of a data flow modeling a data mapping, comprising:
receive a plurality of mapping fragments for inclusion in the data flow, wherein each mapping fragment comprises a plurality of por

Assignees

• Informatica Llc

Inventors

• Chow Jedidiah Jd
• Gannon Kathleen L
• Kranz Itai
• Patnaik Arun
• Sethiya Amit
• Sundaramurthy Lalitha

Classifications

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title

• G06F3/0486

  Drag-and-drop · CPC title

• G06F16/212

  with details for data modelling support · CPC title

• G06F16/25

  Integrating or interfacing systems involving database management systems · CPC title

• G06F8/34 Primary

  Graphical or visual programming · CPC title