Transaction consistency query support for replicated data from recovery log to external data stores

Patent metadata

Field	Value
Publication number	US-11455217-B2
Application number	US-201715850454-A
Country	US
Kind code	B2
Filing date	Dec 21, 2017
Priority date	Jun 3, 2016
Publication date	Sep 27, 2022
Grant date	Sep 27, 2022

Abstract

Transaction consistency query support is available for replicated data from recovery log to external data stores. An external data store is populated with records using entries of a change data table. The change data table has entries for each transaction that has committed and is to be replicated, and each of the entries stores information for each log entry in a recovery log from a database management system. Each log entry identifies a transactional change of data and a transaction completion indicator of one of commit and abort. In response to receiving a query about a transaction of the transactions, a set of records are retrieved from the external data store for the transaction. From the set of records, records whose sequence identifier values are larger than a maximum transaction commit sequence identifier are removed. From the set of records, remaining records having transaction consistency are returned.

First claim

What is claimed is:

1. A computer-implemented method for transaction consistency query support for replicated data from a recovery log to an external data store, comprising operations for:
populating a replicated transformation table in an external data store with records using entries of a change data table in a database management system, by:
with sequential publishing, for each of the entries:
determining whether that entry is a last entry for a transaction by checking a last transaction message indicator; and
in response to determining that the entry is the last entry for the transaction, setting a maximum transaction commit sequence identifier value for the transaction to a sequence identifier of the last entry; and

with parallel publishing, for each of the entries,
determining whether that entry is the last entry for the transaction by checking the last transaction message indicator; and
in response to determining that the entry is the last entry for the transaction, setting a maximum transaction commit sequence identifier value to a last consecutive applied sequence identifier; and

in response to receiving a query at the external data store,
retrieving a set of records from the replicated transformation table in the external data store;
determining whether partial changes have been applied, wherein the partial changes are in records of in-flight transactions that have not committed yet in the external data store;
in response to determining that the partial changes have been applied,
removing, from the set of records, the records of the in-flight transactions whose sequence identifier values are larger than the maximum transaction commit sequence identifier value; and
returning, from the set of records, remaining records having transaction consistency; and

in response to determining that the partial changes have not been applied, returning the set of records having transaction consistency.

2. The computer-implemented method of claim 1 , wherein, in response to receiving an abort transaction completion indicator, discarding the entries for the transaction from an in-memory queue.

3. The computer-implemented method of claim 1 , wherein the remaining records contain records for committed transactions and do not contain records for the in-flight transactions.

4. The computer-implemented method of claim 1 , wherein information from a log entry includes a sequence identifier, a commit time, and a data loss indictor, wherein the sequence identifier and the commit time are used to determine a source commit time for which transactions to that point have been inserted into the external data store, and wherein the sequence identifier and the data loss indicator are used to detect data loss and corruption.

5. The computer-implemented method of claim 1 , wherein a Software as a Service (SaaS) is configured to perform the operations of the method.

6. The computer-implemented method of claim 1 , wherein the change data table is populated by:
storing information from log entries of a recovery log into entries of an in-memory queue;
in response to receiving a commit transaction completion indicator, reconstructing a transaction by grouping entries of the in-memory queue that have a same transaction identifier; and
storing the grouped entries for the reconstructed transaction in the change data table.

Assignees

• IBM

Inventors

• Bourbonnais Serge
• D'Costa Austin F M
• Lau Yat On
• Li Xiao
• Min Hong
• Su Gong
• Wierenga Jonathan W
• Zentgraf Christian
• Zhang Kan

Classifications

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F2201/82

  Solving problems relating to consistency · CPC title

• G06F2201/80

  Database-specific techniques · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F11/1474 Primary

  in transactions (updating of structured data in databases G06F16/23) · CPC title