Graphic representations of data relationships

Patent metadata

Field	Value
Publication number	US-9760612-B2
Application number	US-39376509-A
Country	US
Kind code	B2
Filing date	Feb 26, 2009
Priority date	Feb 26, 2008
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

Presenting a diagram indicating relationships among data items stored in a data management system includes: receiving a request that identifies a first data item stored in the data management system from a user interface; retrieving stored configuration information that includes a plurality of selection specifications for selecting data items in the data management system that are related to a given data item of a predetermined type, where each selection specification is associated with a different respective predetermined type; querying the data management system to identify a set of one or more data items according to a selection specification from the configuration information that is associated with a type of the first data item; for each of multiple returned data items in the identified set, querying the data management system to determine whether additional data items are identified according to a selection specification from the configuration information that is associated with a type of the returned data item; generating a diagram indicating relationships among data items identified using the configuration information; and presenting the generated diagram over the user interface.

First claim

What is claimed is:

1. A method for presenting a data lineage diagram indicating relationships among metadata items stored in a metadata management system accessible to a computing system, the method including:
receiving a request that identifies a first metadata item stored in the metadata management system from a user interface;
retrieving at least a first configuration information set from storage that stores multiple configuration information sets, where each configuration information set includes a plurality of selection specifications for executing queries from the computing system to select metadata items in the metadata management system that are related to a given metadata item of a predetermined type, where each configuration information set in the storage includes different selection specifications, and where each selection specification from the first configuration information set is associated with a different respective predetermined type;
querying the metadata management system using one or more selection specifications from the first configuration information set to identify a set of one or more metadata items that are related to the first metadata item, where a first selection specification from the first configuration information set is associated with a type of the first metadata item; and
generating a data lineage diagram indicating data lineage relationships among programs and data represented by the metadata items identified using the first configuration information set, the data lineage diagram including nodes that correspond to the programs and data represented by the metadata items and edges that represent the data lineage relationships among the programs and data represented by the metadata items.

2. The method of claim 1 , wherein querying the metadata management system to identify the set of one or more metadata items that are related to the first metadata item includes dynamically formulating a query using the first selection specification in response to the request, processing the query by the metadata management system, and receiving a query result from the metadata management system identifying the set of one or more metadata items.

3. The method of claim 2 , wherein the query includes at least one navigation action that navigates the metadata management system to retrieve metadata items that are referenced by an attribute of the first metadata item.

4. The method of claim 2 , wherein the query includes at least one navigation action that navigates the metadata management system to retrieve metadata items that have an attribute referencing the first metadata item.

5. The method of claim 2 , wherein processing the query by the metadata management system includes navigating the metadata management system recursively and returning at least some metadata items found at each of multiple steps of the recursion.

6. The method of claim 1 , further including:
for each of multiple returned metadata items in the identified set, querying the metadata management system to identify additional metadata items according to a selection specification from the first configuration information set that is associated with a type of the returned metadata item, wherein querying the metadata management system to identify additional metadata items includes performing multiple iterations of queries, where each iteration includes identifying another set of one or more metadata items each related to at least one metadata item of a previous set of one or more metadata items from a previous iteration.

7. The method of claim 6 , wherein iterations of queries are performed until no more metadata items are found that are related to any of the previous sets of one or more metadata items.

8. The method of claim 1 , further including receiving from the user interface a selection of a configuration file including the first configuration information set from multiple stored configuration files.

9. The method of claim 1 , wherein the first configuration information set includes a label specification associated with the type of the first metadata item for constructing a label for a node of the generated data lineage diagram representing the first metadata item.

10. The method of claim 1 , wherein the first configuration information set includes a label specification associated with the type of the first metadata item for constructing a label for an edge of the generated data lineage diagram representing a relationship between the first metadata item and another metadata item.

11. The method of claim 1 , wherein each predetermined type corresponds to a different portion of a dataflow graph processing system.

12. The method of claim 11 , wherein at least one predetermined type corresponds to a metadata item representing a portion of a dataflow graph that includes nodes representing processing components and links representing flows of data between the processing components.

13. The method of claim 12 , wherein at least one predetermined type corresponds to a metadata item representing a dataflow graph.

14. The method of claim 13 , wherein at least one predetermined type corresponds to a metadata item representing a component within the dataflow graph.

15. The method of claim 11 , wherein at least one predetermined type corresponds to a metadata item representing a field in a dataset.

16. The method of claim 1 , wherein the first metadata item includes metadata describing stored data that corresponds to at least a portion of a dataset or an executable program.

17. The method of claim 16 , wherein the stored data is stored in a first data source different from the metadata management system, and the generated data lineage diagram includes an edge representing a data lineage relationship between a node representing the stored data and a node representing data that corresponds to at least a portion of a dataset or an executable program stored in a second data source different from the first data source and the metadata management system.

18. The method of claim 16 , wherein the generated data lineage diagram includes an edge representing a downstream data lineage relationship between a node representing the stored data and a node representing an entity affected by the stored data.

19. The method of claim 16 , wherein the generated data lineage diagram includes an edge representing an upstream data lineage relationship between a node representing the stored data and a node representing a source from which the stored data is derived.

20. The method of claim 11 , wherein at least one predetermined type corresponds to a metadata item representing a dataset.

21. The method of claim 1 , wherein the selection specifications each specify at least one selection action to navigate among and select metadata items stored in the metadata management system.

22. The method of claim 1 , further including:
for each of multiple returned metadata items in the identified set, querying the metadata management system to identify additional metadata items according to a selection specification from the first configuration information set that is associated with a type of the returned metadata item.

23. The method of claim 1 , further including:
presenting the generated data lineage diagram over the user interface.

24. The method of claim 1 , wherein retrieving at least the first configuration information set includes retrieving the first configuration information set associated wit

Assignees

• Brainerd Jeffrey
• Morse Alan
• Ab Initio Technology Llc

Inventors

• Brainerd Jeffrey
• Morse Alan

Classifications

• G06F17/30554 Primary

  Physics · mapped topic

• G06F17/30994

  Physics · mapped topic

• Y02D30/70

  in wireless communication networks · CPC title

• G06F17/40 Primary

  Data acquisition and logging (for input to computer G06F3/00) · CPC title

• G06F16/2425

  Iterative querying; Query formulation based on the results of a preceding query · CPC title