Systems and methods for automated data migration

Patent metadata

Field	Value
Publication number	US-2023052786-A1
Application number	US-202217977504-A
Country	US
Kind code	A1
Filing date	Oct 31, 2022
Priority date	Aug 20, 2020
Publication date	Feb 16, 2023
Grant date	—

Abstract

A data movement system is provided for moving data using a data-to-file-to-data movement path. The data movement system includes a source database, a target database, a configuration database, and a data movement server. The data movement is in communication with the source database, the target database, and the configuration database. The processor is configured to receive a configuration record including source details and target details. The processor is also configured to define an extraction query based on the source details and to apply the extraction query to the source database to obtain an extraction load. The processor is further configured to generate a load file based on the extraction load, to define a load script based on the target details, and to apply the load script to the load file to obtain a load query. The processor is also configured to update the target database with the load query.

First claim

What is claimed is:

1 . A data movement system for moving data using a data-to-file-to-data movement path, comprising:
a source database; a target database; a configuration database; a data movement server comprising a processor and a memory, the data movement server in communication with the source database, the target database, and the configuration database, the processor configured to: facilitate movement of data from the source database to an intermediary file, by:
receiving a configuration record, including a plurality of source details and a plurality of target details;
defining an extraction query based on the source details;
applying the extraction query to the source database to obtain an extraction load, wherein the data comprises the extraction load; and
generating a load file based on the extraction load, wherein the intermediary file comprises the load file; and
facilitate movement of the data from the intermediary file to the target database, by:
defining a load script based on the target details;
applying the load script to the load file to obtain a load query; and
running the load query to update the target database using the load file.

2 . The data movement system of claim 1 , wherein the processor is further configured to:
identify a plurality of extracted elements associated with the source details; identify a size associated with each extracted element; logically define the extraction query as a plurality of extraction sub-queries defined based on the sizes associated with each extracted element; and apply the defined plurality of extraction sub-queries to update the target database in parallel processes.

3 . The data movement system of claim 1 , wherein the processor is further configured to:
identify a plurality of updated elements associated with the target details; identify a size associated with each updated element; logically define the load query as a plurality of load sub-queries defined based on the associated sizes of each updated element; and apply the defined plurality of load sub-queries to obtain the extraction load and generate the load file in parallel processes.

4 . The data movement system of claim 1 , wherein the processor is further configured to:
receive the configuration record from a metadata table within a configuration table, wherein the configuration record further includes an active flag and a near real-time data flag associated with the source details and the target details.

5 . The data movement system of claim 1 , wherein the processor is further configured to:
upon initiating updating of the target database with the load query, update an audit table within the configuration database with a load start time; and upon completing the updating of the target database with the load query, update the audit table with a load end time.

6 . The data movement system of claim 1 , wherein the processor is further configured to:
upon initiating applying the extraction query, update an audit table within the configuration database with an extraction start time; and upon completing applying the extraction query, update the audit table with an extraction end time.

7 . The data movement system of claim 1 , wherein the processor is further configured to:
receive source parameters identifying a source server and a source database; and generate the extraction query, in part, based on the source parameters.

8 . The data movement system of claim 1 , wherein the processor is configured to define the extraction query to extract the data to provide near-real time data availability upon loading.

9 . A method for moving data from a source database to a target database using a data-to-file-to-data movement path implemented by a data movement server, the data movement server including a processor and a memory, the data movement server in communication with a source database, a target database, and a configuration database, the method comprising:
facilitating movement of data from the source database to an intermediary file, by:
receiving a configuration record, including a plurality of source details and a plurality of target details;
defining an extraction query based on the source details;
applying the extraction query to the source database to obtain an extraction load, wherein the data comprises the extraction load; and
generating a load file based on the extraction load, wherein the intermediary file comprises the load file; and
facilitating movement of the data from the intermediary file to the target database, by:
defining a load script based on the target details;
applying the load script to the load file to obtain a load query; and
running the load query to update the target database using the load file.

10 . The method of claim 9 , further comprising:
identifying a plurality of extracted elements associated with the source details; identifying a size associated with each extracted element; logically defining the extraction query as a plurality of extraction sub-queries defined based on the associated sizes of each extracted element; and applying the defined extraction sub-queries to update the target database in parallel processes.

11 . The method of claim 9 , further comprising:
identifying a plurality of updated elements associated with the target details; identifying a size associated with each updated element; logically defining the load query as a plurality of load sub-queries defined based on the associated sizes of each updated element; and applying the defined load sub-queries to obtain the extraction load and generate the load file in parallel processes.

12 . The method of claim 9 , further comprising:
upon initiating updating of the target database with the load query, updating an audit table within the configuration database with a load start time; and upon completing the updating of the target database with the load query, updating the audit table with a load end time.

13 . The method of claim 9 , further comprising:
upon initiating applying the extraction query, updating an audit table within the configuration database with an extraction start time; and upon completing applying the extraction query, updating the audit table with an extraction end time.

14 . The method of claim 9 , further comprising:
receiving source parameters identifying a source server and a source database; and generating the extraction query, in part, based on the source parameters.

15 . A data movement server for moving data from a source database to a target database using a data-to-file-to-data movement path, the data movement server comprising a processor and a memory, the data movement server in communication with the source database, the target database, and a configuration database, the processor is configured to:
facilitate movement of data from the source database to an intermediary file, by:
receiving a configuration record, including a plurality of source details and a plurality of target details;
defining an extraction query based on the source details; applying the extraction query to the source database to obtain an extraction load, wherein the data comprises the extraction load; and generating a load file based on the extraction load, wherein the intermediary file comprises the load file; and facilitate movement of the data from the intermediary file to the target database, by:
defining a load script based on the target details;
applying the load script to the load file to obtain a load query; and
running the load query to update the target database using t

Assignees

• Cigna Intellectual Property Inc

Inventors

• Bose Saktheeswaran
• Kumar Manickavasagam

Classifications

• G06F16/2455

  Query execution · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F16/214 Primary

  Database migration support · CPC title

• G06F16/254 Primary

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title