Automated data exploration and validation

Patent metadata

Field	Value
Publication number	US-10423631-B2
Application number	US-201715405607-A
Country	US
Kind code	B2
Filing date	Jan 13, 2017
Priority date	Jan 13, 2017
Publication date	Sep 24, 2019
Grant date	Sep 24, 2019

Abstract

Embodiments for automated data exploration and validation by a processor. One or more optimal data flows are provided in response to a query for one or more heterogeneous data sources according to an inference model based on a knowledge graph of heterogeneous data source relationships, a plurality of data flows between one or more heterogeneous data sources relating to the query, and an ontology of concepts and representing a domain knowledge of the one or more heterogeneous data sources.

First claim

The invention claimed is:

1. A method, by a processor, for automated data exploration and validation, comprising:
providing one or more optimal data flows in response to a query for one or more heterogeneous data sources according to an inference model based on a knowledge graph of heterogeneous data source relationships, a plurality of data flows between one or more heterogeneous data sources relating to the query, and an ontology of concepts and representing a domain knowledge of the one or more heterogeneous data sources;
providing an analytical flow for one or more of the plurality of data flows for those of the one or more heterogeneous data sources that are undetected;
aggregating or dis aggregating two or more of the one or more of the plurality of data flows for the one or more heterogeneous data sources that are nested within the knowledge graph; and
receiving one or more criteria from a user via an interactive graphical user interface (GUI) to use for defining the one or more optimal data flows.

2. The method of claim 1 , further including:
measuring one or more key performance indicators (KPIs) of each of the plurality of data flows that answer the query; and
assigning a confidence score to each of the plurality of data flows for each of the plurality of data flows based on the KPIs.

3. The method of claim 2 , further including ranking each of the plurality of data flows according to the confidence score.

4. The method of claim 1 , further including receiving user feedback relating to a confidence score such that the inference model is updated based on the user feedback.

5. The method of claim 1 , further including selecting, as the one or more optimal data flows, at least one of the plurality of data flows having a highest confidence score as compared to those of the plurality of data flows having a lower confidence score in relation to each other according to the inference model.

6. The method of claim 1 , further including providing a mapping between plurality of data flows and the one or more heterogeneous data sources on the knowledge graph that satisfy the query.

7. A system for automated data exploration and validation, comprising: one or more computers with executable instructions that when executed cause the system to: provide one or more optimal data flows in response to a query for one or more heterogeneous data sources according to an inference model based on a knowledge graph of heterogeneous data source relationships, a plurality of data flows between one or more heterogeneous data sources relating to the query, and an ontology of concepts and representing a domain knowledge of the one or more heterogeneous data sources, provide an analytical flow for one or more of the plurality of data flows for those of the one or more heterogeneous data sources that are undetected; aggregate or disaggregate two or more of the one or more of the plurality of data flows for the one or more heterogeneous data sources that are nested within the knowledge graph; and receive one or more criteria from a user via an interactive graphical user interface (GUI) to use for defining the one or more optimal data flows.

8. The system of claim 7 , wherein the executable instructions:
measure one or more key performance indicators (KPIs) of each of the plurality of data flows that answer the query; and
assign a confidence score to each of the plurality of data flows for each of the plurality of data flows based on the KPIs.

9. The system of claim 7 , wherein the executable instructions rank each of the plurality of data flows according to the confidence score.

10. The system of claim 7 , wherein the executable instructions receive user feedback relating to a confidence score such that the inference model is updated based on the user feedback.

11. The system of claim 7 , wherein the executable instructions select, as the one or more optimal data flows, at least one of the plurality of data flows having a highest confidence score as compared to those of the plurality of data flows having a lower confidence score in relation to each other according to the inference model.

12. The system of claim 7 , wherein the executable instructions provide a mapping between plurality of data flows and the one or more heterogeneous data sources on the knowledge graph that satisfy the query.

13. A computer program product for, by a processor, automated data exploration and validation, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising: an executable portion that provides one or more optimal data flow s in response to a query for one or more heterogeneous data sources according to an inference model based on a knowledge graph of heterogeneous data source relationships, a plurality of data flows between one or more heterogeneous data sources relating to the query, and an ontology of concepts and representing a domain knowledge of the one or more heterogeneous data sources;
an executable portion that provides an analytical flow for one or more of the plurality of data flows for those of the one or more heterogeneous data sources that are undetected, an executable portion that aggregates or disaggregates two or more of the one or more of the plurality of data flows for the one or more heterogeneous data sources that are nested within the knowledge graph; and an executable portion that receives one or more criteria from a user via an interactive graphical user interface (GUI) to use for defining the one or more optimal data flows.

14. The computer program product of claim 13 , further including an executable portion that:
measures one or more key performance indicators (KPIs) of each of the plurality of data flows that answer the query;
assigns a confidence score to each of the plurality of data flows for each of the plurality of data flows based on the KPIs; and
ranks each of the plurality of data flows according to the confidence score.

15. The computer program product of claim 13 , further including an executable portion that receives user feedback relating to a confidence score such that the inference model is updated based on the user feedback.

16. The computer program product of claim 13 , further including an executable portion that selects, as the one or more optimal data flows, at least one of the plurality of data flows having a highest confidence score as compared to those of the plurality of data flows having a lower confidence score in relation to each other according to the inference model.

17. The computer program product of claim 13 , further including an executable portion that provides a mapping between plurality of data flows and the one or more heterogeneous data sources on the knowledge graph that satisfy the query.

Assignees

• IBM

Inventors

• Fischer Ulrike
• Fusco Francesco
• Pompey Pascal
• Sinn Mathieu

Classifications

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F16/24578 Primary

  using ranking · CPC title

• G06F16/248

  Presentation of query results · CPC title

• G06F17/5009

  Physics · mapped topic