Recovery log analytics with a big data management platform

Patent metadata

Field	Value
Publication number	US-10216582-B2
Application number	US-201615237407-A
Country	US
Kind code	B2
Filing date	Aug 15, 2016
Priority date	Aug 15, 2016
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

Provided are techniques for replicating relational transactional log data to a big data platform. Change records contained in change data tables are fetched. A relational change history with transaction snapshot consistency is rebuilt to generate consistent change records by joining the change data tables and a unit of work table based on a commit sequence identifier. The consistent change records are stored on the big data platform, and queries are answered on the big data platform using the consistent change records.

First claim

What is claimed is:

1. A computer program product for replicating relational transactional log data to a big data platform, the computer program product comprising a non-transitory computer readable storage medium having program code embodied therewith, the program code executable by at least one processor to perform operations comprising:
fetching change records contained in change data tables;
rebuilding a relational change history with transaction snapshot consistency to generate consistent change records by joining the change data tables and a unit of work table based on a commit sequence identifier, wherein the rebuilding is performed by one of a relational database management system and the big data platform; and
storing the consistent change records on the big data platform, wherein queries are answered on the big data platform using the consistent change records.

2. The computer program product of claim 1 , wherein fetching the change records is performed using one of centralized parallel transmission and decentralized parallel transmission.

3. The computer program product of claim 1 , wherein the change data tables are pruned.

4. The computer program product of claim 1 , wherein analysis of the consistent change records is performed on the big data platform.

5. The computer program product of claim 1 , wherein a Software as a Service (SaaS) is configured to perform computer program product operations.

6. The computer program product of claim 1 , wherein the unit of work table is used to determine whether there are new records for new transactions in the change data tables.

7. The computer program product of claim 1 , wherein metadata is stored that includes a source schema name, a table name of a table, subscribed column names, and a subscription state of the table.

8. A computer system for replicating relational transactional log data to a big data platform, comprising:
one or more processors, one or more computer-readable memories and one or more computer-readable, tangible storage devices; and
program instructions, stored on at least one of the one or more computer-readable, tangible storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to perform operations comprising:
fetching change records contained in change data tables;
rebuilding a relational change history with transaction snapshot consistency to generate consistent change records by joining the change data tables and a unit of work table based on a commit sequence identifier, wherein the rebuilding is performed by one of a relational database management system and the big data platform; and
storing the consistent change records on the big data platform, wherein queries are answered on the big data platform using the consistent change records.

9. The computer system of claim 8 , wherein fetching the change records is performed using one of centralized parallel transmission and decentralized parallel transmission.

10. The computer system of claim 8 , wherein the change data tables are pruned.

11. The computer system of claim 8 , wherein analysis of the consistent change records is performed on the big data platform.

12. The computer system of claim 8 , wherein a Software as a Service (SaaS) is configured to perform system operations.

13. The computer system of claim 8 , wherein the unit of work table is used to determine whether there are new records for new transactions in the change data tables.

14. The computer system of claim 8 , wherein metadata is stored that includes a source schema name, a table name of a table, subscribed column names, and a subscription state of the table.

Assignees

• IBM

Inventors

• Bourbonnais Serge
• D'Costa Austin F M
• Lau Yat On
• Li Xiao
• Min Hong
• Su Gong
• Wierenga Jonathan W
• Zentgraf Christian

Classifications

• G06F2201/82

  Solving problems relating to consistency · CPC title

• G06F11/1451 Primary

  by selection of backup contents · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/215 Primary

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title