Archiving a table in a data analysis system

Patent metadata

Field	Value
Publication number	US-12443625-B2
Application number	US-202318303810-A
Country	US
Kind code	B2
Filing date	Apr 20, 2023
Priority date	Mar 22, 2023
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

An embodiment for archiving data in a data analysis system. The embodiment may receive a request to archive a source partition of a source table of source database system of the data analysis system. The embodiment may determine whether the source table is partitioned into partitions by ranges of an attribute of the source table. The embodiment may, in response to determining that the source table is partitioned by range, lock the source partition to prevent changes of the source partition. The embodiment may determine whether the source partition is synchronized with a corresponding partition in the target table. In response to determining that the source partition is synchronized with the corresponding partition in the target table, the embodiment may remove data in the source partition from the source table.

First claim

The invention claimed is:

1. A method for archiving data in a data analysis system, the data analysis system comprising a source database system and a target database system, the source database system comprising a source table, the source table being partitioned into source partitions by ranges of an attribute of the source table, the target database system comprising a target table corresponding to the source table, wherein a data synchronization in the data analysis system enables a replication of data of the source table from the source database system to the target database system; the method comprising:
receiving a request to archive a source partition of the source table;
determining whether the source table is partitioned into partitions by ranges of an attribute of the source table;
in response to determining that the source table is partitioned by range, locking the source partition to prevent changes of the source partition;
determining whether the source partition is synchronized with a corresponding partition in the target table;
in response to determining that the source partition is synchronized with the corresponding partition in the target table, removing data of the source partition from the source table;
receiving a query referencing a whole range of the source table;
offloading the received query to the target database system for executing the received query at the target database system on the target table that comprises archived and non-archived partitions of the source table; and
creating at the target database system a first database view representing the non-archived partitions, and a second database view representing the archived and the non-archived partitions, wherein execution of the received query is performed using the second database view.

2. The method of claim 1 , wherein each of the first and the second database views are associated with predicates to indicate whether each of the database views has archived partitions, wherein the execution of the received query is performed using the predicates.

3. The method of claim 1 , wherein the first database view is created upon receiving an archive request and before creating the second database view.

4. The method of claim 1 , wherein the second database view is created in response to removing data of the source partition.

5. The method of claim 1 , wherein determining whether the source partition is synchronized with the corresponding partition of the target table further comprises:
determining whether the data synchronization between the source database system and the target database system is automatically enabled;
in response to determining that the automatic data synchronization between the source database system and the target database system is enabled, determining whether all changes made to the source table until the request is received are propagated to the target database system; and
in response to determining that the automatic data synchronization between the source database system and the target database system is not enabled, loading the source partition to the target database system, thereby determining that the source partition is synchronized with the corresponding partition of the target table.

6. The method of claim 5 , wherein determining whether all the changes made until the request is received are propagated to the target database system further comprises:
sending a timestamp of a latest commit of the source table to the target database system;
receiving a synchronization confirmation from the target database system in response to sending the timestamp; and
in response to receiving the synchronization confirmation, determining that the source table is synchronized with the target table.

7. The method of claim 1 , further comprising:
receiving a query referencing non-archived partitions of the source table; and
executing the received query in the source database system on the non-archived partitions or offloading the received query to the target database system for executing the received query at the target database system on the non-archived partitions.

8. The method of claim 1 , wherein locking the source partition to prevent changes of the source partition further comprises:
creating an image copy of the source partition for enabling a recovery of the source table.

9. The method of claim 1 , wherein removing the data of the source partition from the source table further comprises:
performing a purge operation of the source partition;
unlocking the source partition; and
marking the source partition as a read-only partition.

10. The method of claim 1 , wherein the source database system comprises an online transaction processing (OLTP) system, and the target database system comprises one or more online analytical processing (OLAP) systems.

11. A computer program product for archiving data in a data analysis system, the data analysis system comprising a source database system and a target database system, the source database system comprising a source table, the source table being partitioned into source partitions by ranges of an attribute of the source table, the target database system comprising a target table corresponding to the source table, wherein a data synchronization in the data analysis system enables a replication of data of the source table from the source database system to the target database system, the computer program product comprising:
one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media that when executed by a processor perform the operations of:
receiving a request to archive a source partition of the source table;
determining whether the source table is partitioned into partitions by ranges of an attribute of the source table;
in response to determining that the source table is partitioned by range, locking the source partition to prevent changes of the source partition;
determining whether the source partition is synchronized with a corresponding partition in the target table;
in response to determining that the source partition is synchronized with the corresponding partition in the target table, removing data of the source partition from the source table;
receiving a query referencing a whole range of the source table;
offloading the received query to the target database system for executing the received query at the target database system on the target table that comprises archived and non-archived partitions of the source table; and
creating at the target database system a first database view representing the non-archived partitions, and a second database view representing the archived and the non-archived partitions, wherein execution of the received query is performed using the second database view.

12. The computer program product of claim 11 , wherein each of the first and the second database views are associated with predicates to indicate whether each of the database views has archived partitions, wherein the execution of the received query is performed using the predicates.

13. The computer program product of claim 11 , wherein the first database view is created upon receiving an archive request and before creating the second database view.

14. The computer program product of claim 11 , wherein the second database view is created in response to removing data of the source partition.

15. A computer system for archiving data in a data analysis system, the data analysis system comprising a source database system and a target database system, the source database syst

Assignees

• IBM

Inventors

• Simanjuntak Manogari Nogi
• Kameswaran Sowmya
• Broll Björn
• Martin Daniel

Classifications

• G06F16/24554

  Unary operations; Data partitioning operations · CPC title

• G06F16/2445 Primary

  Data retrieval commands; View definitions · CPC title

• G06F16/215

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title

• G06F16/2343

  Locking methods, e.g. distributed locking or locking implementation details · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title