Systems and methods for traceability of data changes

Patent metadata

Field	Value
Publication number	US-2024289310-A1
Application number	US-202318173917-A
Country	US
Kind code	A1
Filing date	Feb 24, 2023
Priority date	Feb 24, 2023
Publication date	Aug 29, 2024
Grant date	—

Abstract

In one aspect, an example methodology implementing the disclosed techniques includes, by a computing device, receiving information about a data element affected by a data change and generating a data lineage of the data element, wherein the data lineage indicates relationships between the data element and one or more other data elements, the one or more other data elements indicated in the received information about the data element. The method also includes, by the computing device, generating a lineage template for the data element based on data lineage information, wherein the lineage template defines a workflow for the data element to complete the data change in the data lineage, and generating a lineage graph based on the data lineage information, wherein the lineage graph tracks the data lineage of the data element. The method may further include monitoring performance of the workflow using the lineage graph.

First claim

What is claimed is:

1 . A method comprising:
receiving, by a computing device, information about a data element affected by a data change; generating, by the computing device, a data lineage of the data element, wherein the data lineage indicates relationships between the data element and one or more other data elements, the one or more other data elements indicated in the received information about the data element; generating, by the computing device, a lineage template for the data element based on data lineage information, wherein the lineage template defines a workflow for the data element to complete the data change in the data lineage; and generating, by the computing device, a lineage graph based on the data lineage information, wherein the lineage graph tracks the data lineage of the data element.

2 . The method of claim 1 , wherein the information about the data element includes identification and versioning information and referentiality information of the data element.

3 . The method of claim 2 , wherein the one or more other data elements being indicated in the referentiality information.

4 . The method of claim 1 , wherein the lineage template defines one or more actions for the workflow, wherein an action of the one or more actions having values that indicate a sequencing of the action, an average time to perform the action, a minimum time to perform the action, a maximum time to perform the action, and a timeout to generate an alert.

5 . The method of claim 1 , wherein the lineage template provides for indicating creation, completion, and status of data tasks.

6 . The method of claim 1 , further comprising, monitoring performance of the workflow using the lineage graph.

7 . The method of claim 6 , wherein the lineage graph shows that the workflow is partially completed or completed.

8 . The method of claim 1 , wherein the generating the data lineage includes updating an existing data lineage of the data element.

9 . The method of claim 1 , wherein the generating the lineage template includes identifying an existing lineage template generated for the data element.

10 . A system comprising:
one or more non-transitory machine-readable mediums configured to store instructions; and one or more processors configured to execute the instructions stored on the one or more non-transitory machine-readable mediums, wherein execution of the instructions causes the one or more processors to carry out a process comprising:
receiving information about a data element affected by a data change;
generating a data lineage of the data element, wherein the data lineage indicates relationships between the data element and one or more other data elements, the one or more other data elements indicated in the received information about the data element;
generating a lineage template for the data element based on data lineage information, wherein the lineage template defines a workflow for the data element to complete the data change in the data lineage; and
generating a lineage graph based on the data lineage information, wherein the lineage graph tracks the data lineage of the data element.

11 . The system of claim 10 , wherein the information about the data element includes identification and versioning information and referentiality information of the data element.

12 . The system of claim 11 , wherein the one or more other data elements being indicated in the referentiality information.

13 . The system of claim 10 , wherein the lineage template defines one or more actions for the workflow, wherein an action of the one or more actions having values that indicate a sequencing of the action, an average time to perform the action, a minimum time to perform the action, a maximum time to perform the action, and a timeout to generate an alert.

14 . The system of claim 10 , wherein the process further comprises monitoring performance of the workflow using the lineage graph.

15 . The system of claim 14 , wherein the lineage graph shows that the workflow is completed.

16 . The system of claim 14 , wherein the lineage graph shows that the workflow is partially completed.

17 . A non-transitory machine-readable medium encoding instructions that when executed by one or more processors cause a process to be carried out, the process including:
receiving information about a data element affected by a data change; generating a data lineage of the data element, wherein the data lineage indicates relationships between the data element and one or more other data elements, the one or more other data elements indicated in the received information about the data element; generating a lineage template for the data element based on data lineage information, wherein the lineage template defines a workflow for the data element to complete the data change in the data lineage; and generating a lineage graph based on the data lineage information, wherein the lineage graph tracks the data lineage of the data element.

18 . The machine-readable medium of claim 17 , wherein the information about the data element includes identification and versioning information and referentiality information of the data element, the one or more other data elements being indicated in the referentiality information.

19 . The machine-readable medium of claim 17 , wherein the lineage template defines one or more actions for the workflow, wherein an action of the one or more actions having values that indicate a sequencing of the action, an average time to perform the action, a minimum time to perform the action, a maximum time to perform the action, and a timeout to generate an alert.

20 . The machine-readable medium of claim 17 , wherein the process further comprises monitoring performance of the workflow using the lineage graph.

Assignees

• Dell Products Lp

Inventors

• Ooi Lee Chuen
• Gopalakrishnan Harikrishnan Nair

Classifications

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F11/3419

  by assessing time · CPC title

• G06F16/219 Primary

  Managing data history or versioning (querying versioned data G06F16/2474; querying temporal data G06F16/2477) · CPC title