Interactive tree representing attribute quality or consumption metrics for data ingestion and other applications

Patent metadata

Field	Value
Publication number	US-12340333-B2
Application number	US-202217693778-A
Country	US
Kind code	B2
Filing date	Mar 14, 2022
Priority date	Mar 14, 2022
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

Embodiments provide systems, methods, and computer storage media for management, assessment, navigation, and/or discovery of data based on data quality, consumption, and/or utility metrics. Data may be assessed using attribute-level and/or record-level metrics that quantify data: “quality”—the condition of data (e.g., presence of incorrect or incomplete values), its “consumption”—the tracked usage of data in downstream applications (e.g., utilization of attributes in dashboard widgets or customer segmentation rules), and/or its “utility”—a quantifiable impact resulting from the consumption of data (e.g., revenue or number of visits resulting from marketing campaigns that use particular datasets, storage costs of data). This data assessment may be performed at different stages of a data intake, preparation, and/or modeling lifecycle. For example, an interactive tree view may visually represent a nested attribute schema and attribute quality or consumption metrics to facilitate discovery of bad data before ingesting into a data lake.

First claim

What is claimed is:

1. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
receiving input identifying a dataset for ingestion into a data lake, the dataset including a plurality of attributes;
generating, based on sample data from the dataset, a first attribute quality metric for a first attribute of the plurality of attributes of the dataset, wherein the first attribute quality metric indicates at least one of: cardinality, completeness, correctness, and objectivity; and
causing a user interface to present an interactive tree view with a hierarchy of a plurality nodes that represent a nested attribute schema of the dataset, a first node of the plurality of the nodes visually representing quality of the first attribute based on the quality metric for the the first attribute, where an interaction with the first attribute displayed in the interactive tree causes the user interface to be updated to display a visual representation of the value of the first attribute.

2. The one or more computer storage media of claim 1 , the operations further comprising ingesting the sample data from the dataset into a landing zone separate from the data lake.

3. The one or more computer storage media of claim 1 , the operations further comprising generating the first attribute quality metric off the sample data stored in a landing zone separate from the data lake.

4. The one or more computer storage media of claim 1 , the operations further comprising generating the first attribute quality metric off a stream of the sample data being ingested into in a landing zone separate from the data lake.

5. The one or more computer storage media of claim 1 , the operations further comprising, based on a second interaction with a second node of the plurality of nodes, causing the user interface to present values of multiple of the corresponding to a plurality of attribute quality metrics quantifying different measures of quality of the corresponding attribute of the plurality of attributes.

6. The one or more computer storage media of claim 1 , wherein the first node of the plurality of nodes is associated with a glyph that visually represents a value of a combined attribute quality metric that quantifies health of the corresponding attribute based on a combination of a plurality of attribute quality metrics for the corresponding attribute including the first attribute quality metric.

7. The one or more computer storage media of claim 1 , the operations further comprising, based on input indicating an instruction to ingest the dataset responsive to the interactive tree view, ingesting the dataset into the data lake.

8. The one or more computer storage media of claim 1 , the operations further comprising accepting input configuring which of a plurality of attribute quality metrics the nodes in the interactive tree view represent.

9. A method comprising:
receiving input identifying a dataset for ingestion into a data lake, the dataset including a plurality of attributes;
generating, based on sample data from the dataset, a first attribute quality metric for an attribute of the plurality of attributes of the dataset, wherein the first attribute quality metrics indicates at least one of: cardinality, completeness, correctness, and objectivity; and
causing a user interface to present an interactive tree view with a hierarchy of a plurality of nodes that represent a nested attribute schema of the dataset, a first node of the plurality of the nodes selectable to cause presentation of a representation of a value associated with the first attribute quality metric corresponding to the first attribute represented by the node, where an interaction with the first node displayed in the interactive tree causes the user interface to be updated to display a visual representation of the value of the first attribute quality metric.

10. The method of claim 9 , further comprising ingesting the sample data from the dataset into a landing zone separate from the data lake.

11. The method of claim 9 , further comprising generating the first attribute quality metric off the sample data stored in a landing zone separate from the data lake.

12. The method of claim 9 , further comprising generating the first attribute quality metric off a stream of the sample data being ingested into in a landing zone separate from the data lake.

13. The method of claim 9 , further comprising, based on a second interaction with a second node of the plurality of the nodes, causing the user interface to present values of multiple of the attribute quality metrics quantifying different measures of quality of the corresponding attribute of which the first attribute quality metric is a member.

14. The method of claim 9 , wherein the first node of the plurality of nodes is associated with a glyph that visually represents a value of a combined attribute quality metric that quantifies health of the first attribute based on a combination of multiple attribute quality metrics for the first attribute, where the first attribute quality metric is included in the combination.

15. The method of claim 9 , further comprising, based on input indicating an instruction to ingest the dataset responsive to the interactive tree view, ingesting the dataset into the data lake.

16. The method of claim 9 , further comprising accepting input configuring which of a set of attribute quality metrics the nodes in the interactive tree view represent, where the first attribute quality metric is a member of the set of attribute quality metrics.

17. A computer system comprising:
one or more hardware processors and memory configured to provide computer program instructions to the one or more hardware processors; and
a data management tool configured to use the one or more hardware processors to:
receive input identifying a dataset, the dataset including a plurality of attributes;
access a first attribute quality metric that quantify quality of a first attribute of the plurality of attributes of the dataset or a first attribute consumption metric that quantify tracked consumption of the first attribute of the plurality of attributes of the dataset, wherein the attribute quality metrics indicate at least one of: cardinality, completeness, correctness, and objectivity; and
cause a user interface to present an interactive tree view with a hierarchy of a plurality of nodes that visually represent the plurality of attributes and values associated with a set of attribute quality metrics or a set of attribute consumption metrics, where the first attribute quality metric is a member of the set of attribute quality metrics or the first attribute consumption metric is member of the set of attribute consumption metrics and an interaction with the first attribute displayed in the interactive tree causes the user interface to be updated to display a visual representation of a value associated with the first attribute.

18. The computer system of claim 17 , wherein the data management tool is configured to cause, based on the interaction with the first node, the user interface to present values of the first attribute quality metric quantifying different measures of quality of the first attribute and the first attribute consumption metric quantifying different measures of consumption of the first attribute.

19. The computer system of claim 17 , wherein the first node of the plurality of the nodes is associated with a glyph that

Assignees

• Adobe Inc

Inventors

• Narechania Arpit Ajay
• Du Fan
• Sinha Atanu R
• Rossi Ryan A
• Hoffswell Jane Elizabeth
• Guo Shunan
• Koh Eunyee
• Anderson John
• Surange Sonali
• Mahapatra Saurabh
• Holtcamp Vasanthi

Classifications

• G06F16/215

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06Q10/06393 Primary

  Score-carding, benchmarking or key performance indicator [KPI] analysis · CPC title