Generating a trust factor for data in non-relational or relational databases

Patent metadata

Field	Value
Publication number	US-10353930-B2
Application number	US-201615252282-A
Country	US
Kind code	B2
Filing date	Aug 31, 2016
Priority date	Aug 31, 2016
Publication date	Jul 16, 2019
Grant date	Jul 16, 2019

Abstract

A computer-implemented method includes detecting an update to a record in an entity table of a database. At least one of an age score, a lineage score, and a completeness score for the record is calculated, responsive to the update. A trust factor is calculated, by a computer processor, based on the at least one of the age score, the lineage score, and the completeness score for the record. The trust factor indicates a level of trustworthiness of the record. It is decided whether to use data in the record based on the trust factor.

First claim

What is claimed is:

1. A computer-implemented method comprising:
detecting, by a monitor, an update to a record in an entity table of a database;
wherein the entity table comprises a plurality of records, each having a plurality of columns, wherein each record of the entity table describes a corresponding entity;
calculating at least one of an age score, a lineage score, and a completeness score for the record, responsive to the update;
calculating, by a computer processor, a trust factor of the record based on the at least one of the age score, the lineage score, and the completeness score for the record, wherein the trust factor indicates a level of trustworthiness of the record;
wherein the trust factor of the record is calculated based at least in part on a configuration table, which defines one or more configuration parameters used to determine the at least one of the age score, the lineage score, and the completeness score, and wherein the one or more configuration parameters comprise at least one of a base period for calculating a plurality of age scores for the entity table and a maximum weight allowed for calculating a plurality of completeness scores for the entity table; and
deciding whether to use data in the record based on the trust factor.

2. The computer-implemented method of claim 1 , wherein the database is a non-relational database.

3. The computer-implemented method of claim 1 , wherein the calculating at least one of the age score, the lineage score, and the completeness score for the record comprises calculating the age score for the record, and wherein the calculating the age score for the record comprises:
determining an age of the record;
identifying a maximum age of a plurality of records in the entity table;
calculating the age score of the record based on the age of the record and the maximum age of the plurality of records.

4. The computer-implemented method of claim 1 , wherein the calculating at least one of the age score, the lineage score, and the completeness score for the record comprises calculating the lineage score for the record, and wherein the calculating the lineage score for the record comprises:
identifying a data source of the record;
identifying a weight associated with the data source; and
determining the lineage score based on the weight associated with the data source.

5. The computer-implemented method of claim 1 , wherein the calculating at least one of the age score, the lineage score, and the completeness score for the record comprises calculating the completeness score for the record, and wherein the calculating the completeness score for the record comprises:
establishing a synonym table to define families of related columns in the entity table, wherein the synonym table assigns a weight to each column with each family of related columns;
identifying a first family of columns in the entity table, based on the synonym table, wherein the first family of columns comprises a first set of columns deemed to be synonyms to one another;
identifying, based on the synonym table, a corresponding weight for each column in the first set of columns of the first family;
determining which of the first set of columns are filled in the record;
calculating the completeness score based on the corresponding weight for each of the first set of columns that are filled in the record.

6. The computer-implemented method of claim 5 , wherein the calculating the completeness score for the record comprises:
identifying a second family of columns in a synonym table, wherein the second family of columns comprises a second set of columns deemed to be synonyms to one another, wherein the second family is distinct from the first family;
identifying a corresponding weight for each column in the second set of columns of the second family; and
determining which of the second set of columns are filled in the record;
wherein the calculating the completeness score based on the corresponding weight for each of the first set of columns that are filled in the record further comprises basing the completeness score on the corresponding weight for of the second set of columns that are filled in the record.

7. The computer-implemented method of claim 1 , wherein the calculating at least one of the age score, the lineage score, and the completeness score for the record is based on a configuration table, and further comprising:
modifying the configuration table based on historical use of the entity table.

8. The computer-implemented method of claim 1 , wherein the database is a non-relational database, the computer-implemented method further comprising:
determining that a first trust factor of the record of the non-relational database is higher than a second trust factor of a second record of the non-relational database; and
merging the record of the non-relational database and the second record of the non-relational database, based on a determination that the second record is a duplicate of the record, by resolving a conflict between the record and the second record in favor of the record, based on the first trust factor being higher than the second trust factor.

9. A system comprising:
a memory having computer readable instructions; and
one or more processors for executing the computer readable instructions, the computer readable instructions comprising:
detecting an update to a record in an entity table of a database;
wherein the entity table comprises a plurality of records, each having a plurality of columns, wherein each record of the entity table describes a corresponding entity;
calculating at least one of an age score, a lineage score, and a completeness score for the record, responsive to the update;
calculating a trust factor of the record based on the at least one of the age score, the lineage score, and the completeness score for the record, wherein the trust factor indicates a level of trustworthiness of the record;
wherein the trust factor of the record is calculated based at least in part on a configuration table, which defines one or more configuration parameters used to determine the at least one of the age score, the lineage score, and the completeness score, and wherein the one or more configuration parameters comprise at least one of a base period for calculating a plurality of age scores for the entity table and a maximum weight allowed for calculating a plurality of completeness scores for the entity table; and
deciding whether to use data in the record based on the trust factor.

10. The system of claim 9 , wherein the database is a non-relational database.

11. The system of claim 9 , wherein the calculating at least one of the age score, the lineage score, and the completeness score for the record comprises calculating the age score for the record, and wherein the calculating the age score for the record comprises:
determining an age of the record;
identifying a maximum age of a plurality of records in the entity table;
calculating the age score of the record based on the age of the record and the maximum age of the plurality of records.

12. The system of claim 9 , wherein the calculating at least one of the age score, the lineage score, and the completeness score for the record comprises calculating the lineage score for the record, and wherein the calculating the lineage score for the record comprises:
identifying a data source of the record;
identifying a weight associated with the data source; and
determining the lineage score based on the weight associated with the data source.

13. The system of claim 9 , wherein the calculating at least one of the age score, the linea

Assignees

• IBM

Inventors

• Eshwar Bhavani K
• Malla Amit
• Naganna Soma S
• Ramakrishnan Umasuthan

Classifications

• G06F16/288 Primary

  Entity relationship models · CPC title

• G06F16/215 Primary

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title