Creating and processing a data rule for data quality

Patent metadata

Field	Value
Publication number	US-8949166-B2
Application number	US-201113217389-A
Country	US
Kind code	B2
Filing date	Aug 25, 2011
Priority date	Dec 16, 2010
Publication date	Feb 3, 2015
Grant date	Feb 3, 2015

Abstract

A data rule is created and processed by receiving an expression defining a logic of a rule and at least one logical variable, creating a rule definition including the expression and the at least one logical variable for binding each logical variable of the rule with at least one column, associating a characteristic enabling comparison of columns with a first logical variable of the rule definition, and storing the characteristic as part of the rule definition.

First claim

What is claimed is:

1. A computer-implemented method of creating and processing a data rule using an electronic computing device, the method comprising:
receiving an expression defining a logic of a rule and at least one logical variable;
creating a rule definition including the expression and the at least one logical variable for binding each logical variable of the rule with at least one column;
associating a characteristic enabling comparison of columns with a logical variable of the rule definition; and
storing the characteristic as part of the rule definition.

2. The method according to claim 1 , wherein the associating comprises:
receiving information about the binding between the logical variable and a column; and
reading the characteristic from metadata associated with the column.

3. The method according to claim 1 , wherein the associating is achieved by a user and comprises providing the characteristic enabling the comparison of columns.

4. The method according to claim 1 , further comprising:
receiving information about a binding between the logical variable and a column; and
validating the binding based on comparing a characteristic of the column with the stored characteristic.

5. The method according to claim 1 , further comprising:
checking whether a column can be found that has a characteristic similar to the characteristic of the rule definition;
handling a combination of the column and the rule definition as a rule candidate;
testing the rule candidate on a data sample;
removing the rule candidate from a list of rule candidates when the number of failed records exceeds a predefined threshold in the validation step; and
presenting the list of rule candidates to the user for review after the rule candidates have been tested.

6. The method according to claim 1 , further comprising
generating a data rule based on the rule definition; and
selectively applying the data rule to records in a table comprising a column
depending on whether values in the column match the characteristic.

7. A system for creating and processing a data rule comprising:
a processor configured to perform the operations of:
providing a user interface to manage I/O functions of the system, including a rule editor configured to create a plurality of rule definitions including an expression defining a logic of a rule and at least one logical variable for binding each logical variable of the rule with at least one column;
storing within a rule repository the plurality of rule definitions; and
providing a data processing engine in electronic communication with the user interface and the rule repository to associate a characteristic enabling comparison of columns with a logical variable of the rule definition, and to store the characteristic as part of the rule definition in the rule repository.

8. The system according to claim 7 , wherein the processor is further configured to control the data processing engine to receive information about the binding between the logical variable and a column from the user interface, and the processor is further configured to control the data processing engine to read the characteristic from meta-data associated with the column.

9. The system according to claim 7 , wherein the processor is further configured to control the data processing engine to receive the characteristic enabling the comparison of columns from the user interface.

10. The system according to claim 7 , wherein the processor is further configured to control the data processing engine to receive information about a binding between the logical variable and a column from the user interface, and the processor is further configured to control the data processing engine to validate the binding based on comparing characteristic of the column with the stored characteristic.

11. The system according to claim 7 , wherein the processor is further configured to control the data processing engine to check whether a column can be found that has a characteristic similar to the characteristic of the rule definition, to handle a combination of the column and the rule definition as a rule candidate, to test the rule candidate on a data sample, to remove the rule candidate from a list of rule candidates when the number of failed records exceeds a predefined threshold in the validation step, and to present the list of rule candidates to the user interface for review after the rule candidates have been tested.

12. The system according to claim 7 , wherein the processor is further configured to control the data processing engine to generate a data rule based on the rule definition, and to selectively apply the data rule to records in a table comprising a column depending on whether values in the column match the characteristic.

13. A computer program product for creating and processing a data rule comprising:
a computer readable storage device having computer-readable program code embodied therewith, the computer readable program code configured to perform operations of:
receiving an expression defining a logic of a rule and at least one logical variable;
creating a rule definition including the expression and the at least one logical variable for binding each logical variable of the rule with at least one column;
associating a characteristic enabling comparison of columns with a logical variable of the rule definition; and
storing the characteristic as part of the rule definition.

14. The computer program product of claim 13 , wherein the associating comprises:
receiving information about the binding between the logical variable and a column; and
reading the characteristic from metadata associated with the column.

15. The computer program product of claim 13 , wherein the associating is performed by providing the characteristic enabling the comparison of columns.

16. The computer program product of claim 13 , wherein the operations performed by the computer readable program code further comprise:
receiving information about a binding between the logical variable and a column; and
validating the binding based on comparing characteristic of the column with the stored characteristic.

17. The computer program product of claim 13 , wherein the operations performed by the computer readable program code further comprise:
checking whether a column can be found that has a characteristic similar to the characteristic of the rule definition;
handling a combination of the column and the rule definition as a rule candidate;
testing the rule candidate on a data sample;
removing the rule candidate from a list of rule candidates when the number of failed records exceeds a predefined threshold in the validation step; and
presenting the list of rule candidates to the user for review after the rule candidates have been tested.

18. The computer program product of claim 13 , wherein the operations performed by the computer readable program code further comprise:
generating a data rule based on the rule definition; and
selectively applying the data rule to records in a table comprising a column depending upon whether values in the column match the characteristic.

Assignees

• Nelke Sebastian
• Oberhofer Martin
• Saillet Yannick
• Seifert Jens
• IBM

Inventors

• Nelke Sebastian
• Oberhofer Martin
• Saillet Yannick
• Seifert Jens

Classifications

• G06F16/20

  of structured data, e.g. relational data · CPC title

• G06F16/258

  Data format conversion from or to a database · CPC title

• G06N5/025 Primary

  Extracting rules from data · CPC title

• G06F17/30569

  Physics · mapped topic

• G06F17/30286

  Physics · mapped topic