System and method of integrating time-aware data from multiple sources

Patent metadata

Field	Value
Publication number	US-9418124-B2
Application number	US-201314107139-A
Country	US
Kind code	B2
Filing date	Dec 16, 2013
Priority date	Dec 16, 2013
Publication date	Aug 16, 2016
Grant date	Aug 16, 2016

Abstract

A time-aware union operator is disclosed for consistent integration of time-aware data, wherein the time-aware union produces a time-aware consistent integrated view of underlying sources according to specified key constraints and policies. The implementation of time-aware union is idempotent, commutative, and associative, thus making it suitable for data integration, and it produces the same integrated outcome, modulo representation of time, regardless of the order in which sources are integrated.

First claim

The invention claimed is:

1. A computer-implemented method for processing a set of data records having time conflicts, the data records representing respective versions of an entity for which there can be only one preferred value at any given point in time, wherein each of the data records has an n-dimensional time record, the computer-implemented method comprising:
defining a policy from among a plurality of candidate policies, said defined policy designed to resolve time conflicts between those data records having time conflicts;
comparing all data records in a cumulative, pair-wise fashion;
identifying time-based conflicts between pairs of records and identifying time-conflicted pairs;
determining which record in every time-conflicted pair of records is to be adjusted in accordance with said defined policy;
adjusting the time interval of every said determined record in accordance with said defined policy; and
outputting a modified set of data records having said adjusted time intervals, wherein (i) said modified set of data records contains no time conflicts and (ii) said adjusted time intervals in the modified set of data records do not depend on the order in which the data records are processed by said computer-implemented method.

2. The computer-implemented method of claim 1 , wherein said candidate policies depend on at least one of the following: the source of the data records, the n-dimensional time intervals, and values in the data records.

3. The computer-implemented method of claim 1 , wherein said method is implemented as part of a parallel database system.

4. The computer-implemented method of claim 3 , wherein said outputted modified set of data records is one among many intermediate results and said computer-implemented method further comprises the step of merging and outputting all said intermediate results.

5. The computer-implemented method of claim 1 , wherein said outputted modified set of records comprise a concise and complete temporal history of all data records such that a history of said entity is retrievable via a standard XML manipulation language.

6. The computer-implemented method of claim 5 , wherein said standard XML manipulation language is XPath.

7. A system for processing a set of data records having time conflicts, the data records representing respective versions of an entity for which there can be only one preferred value at any given point in time, wherein each of the data records has an n-dimensional time record, the system comprising:
one or more processors; and
a memory comprising instructions which, when executed by the one or more processors, cause the one or more processors to:
define a policy from among a plurality of candidate policies, said defined policy designed to resolve time conflicts between those data records having time conflicts;
compare all data records in a cumulative, pair-wise fashion;
identify time-based conflicts between pairs of records and identify time-conflicted pairs;
determine which record in every time-conflicted pair of records is to be adjusted in accordance with said defined policy;
adjust the time interval of every said determined record to be adjusted in accordance with said defined policy; and
output a modified set of data records having said adjusted time intervals, wherein (i) said modified set of data records contains no time conflicts and (ii) said adjusted time intervals in the modified set of data records do not depend on the order in which the data records are processed by said computer-implemented method.

8. The system of claim 7 , wherein said candidate policies depend on at least one of the following: the source of the data records, the n-dimensional time intervals, and values in the data records.

9. The system of claim 7 , wherein said method is implemented as part of a parallel database system.

10. The system of claim 9 , wherein said outputted modified set of data records is one among many intermediate results and said computer-implemented method further comprises the step of merging and outputting all said intermediate results.

11. The system of claim 7 , wherein said outputted modified set of records comprise a concise and complete temporal history of all data records such that a history of said entity is retrievable via a standard XML manipulation language.

12. The system of claim 11 , wherein said standard XML manipulation language is XPath.

13. A non-transitory computer-readable medium for processing a set of data records having time conflicts, the data records representing respective versions of an entity for which there can be only one preferred value at any given point in time, wherein each of the data records has an n-dimensional time record, the computer-readable medium comprising computer readable program code which, when executed by a computer, cause the computer to computer-implemented method, the medium comprising:
computer readable program code receiving inputs defining a policy from among a plurality of candidate policies, said defined policy designed to resolve time conflicts between those data records having time conflicts;
computer readable program code comparing all data records in a cumulative, pair-wise fashion;
computer readable program code identifying time-based conflicts between pairs of records and identifying time-conflicted pairs;
computer readable program code determining which record in every time-conflicted pair of records is to be adjusted in accordance with said defined policy;
computer readable program code adjusting the time interval of every said determined record in accordance with said defined policy; and
computer readable program code outputting a modified set of data records having said adjusted time intervals, wherein (i) said modified set of data records contains no time conflicts and (ii) said adjusted time intervals in the modified set of data records do not depend on the order in which the data records are processed by said computer-implemented method.

14. The non-transitory computer-readable medium of claim 13 , wherein said candidate policies depend on at least one of the following: the source of the data records, the n-dimensional time intervals, and values in the data records.

15. The non-transitory computer-readable medium of claim 13 , wherein said method is implemented as part of a parallel database system.

16. The non-transitory computer-readable medium of claim 15 , wherein said outputted modified set of data records is one among many intermediate results and said computer-implemented method further comprises the step of merging and outputting all said intermediate results.

17. The non-transitory computer-readable medium of claim 13 , wherein said outputted modified set of records comprise a concise and complete temporal history of all data records such that a history of said entity is retrievable via a standard XML manipulation language.

18. The non-transitory computer-readable medium of claim 17 , wherein said standard XML manipulation language is XPath.

Assignees

• IBM

Inventors

• Roth Mary A
• Tan Wang-Chiew

Classifications

• G06F17/30557 Primary

  Physics · mapped topic

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title

• G06F16/125

  characterised by the use of retention policies (retention policies for HSM systems G06F16/185) · CPC title

• G06F16/258

  Data format conversion from or to a database · CPC title

• G06F16/25 Primary

  Integrating or interfacing systems involving database management systems · CPC title