System and method for cloud-based replication of data

Patent metadata

Field	Value
Publication number	US-12585671-B2
Application number	US-202418819633-A
Country	US
Kind code	B2
Filing date	Aug 29, 2024
Priority date	Aug 5, 2022
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

Embodiments provide systems, methods and computer program products for cloud replication of data. One embodiment includes, accessing a virtual table definition and a data collection definition, the virtual table definition comprising a definition of a set of virtual table fields and a mapping of the set of virtual table fields to a set of target data types; automatically creating a virtual table according to the virtual table definition, the virtual table comprising the set of virtual table fields populated with the data of interest according to the data collection definition; and sending the virtual table and the mapping to a cloud computing environment. One embodiment further includes, in the cloud computing environment, storing the set of virtual table fields as a set of physical table fields in a physical table in a cloud hosted database, the set of physical table fields having the set of target data types.

First claim

What is claimed is:

1 . A system for data replication comprising:
a cloud hosted database; a processor; a memory in communication with the processor, wherein the memory comprises:
a queue;
a message producer, the message producer comprising instructions executable for:
receiving virtual table information for a virtual table and virtual table records for the virtual table, the virtual table information comprising:
a table schema having a virtual table fields;
a mapping of the virtual table fields to target data types;

storing the virtual table information and the virtual table records;
writing references to the virtual table information and the virtual table records to the queue;

a message consumer, the message consumer comprising instructions executable for
dequeuing the references to the virtual table information and the virtual table records from the queue;
based on the references to the virtual table information and the virtual table records, accessing the virtual table information and the virtual table information; and
writing the virtual table records to a physical table that corresponds to the virtual table to replicate data from an on-premises system to the cloud hosted database, wherein at least one of the message producer or the message consumer comprises instructions executable for:
determining if the physical table that corresponds to the virtual table exists in the cloud hosted database; and
based on a determination that the physical table that corresponds to the virtual table does not exist in the cloud hosted database, creating the physical table in the cloud hosted database, wherein a structure of the physical table is defined by the table schema and the mapping of the virtual table fields to the target data types received in the virtual table information.

2 . The system for data replication of claim 1 , wherein the cloud hosted database is accessible by a plurality of cloud services.

3 . The system for data replication of claim 1 , further comprising a persistent storage for storing the virtual table records and the virtual table information.

4 . The system for data replication of claim 3 , wherein the virtual table comprises a first virtual table field for holding a character string representing metadata of a respective file and a second virtual table field for holding the respective file as a binary string.

5 . The system for data replication of claim 3 , wherein the message consumer comprises:
a consumer file service that is executable to access the virtual table information and the virtual table records from the persistent storage; a consumer service executable to dequeue the references to the virtual table information and the virtual table records from the queue and pass the references to the references to the virtual table information and the virtual table records to the consumer file service; and a scheduler executable to schedule the dequeuing of the references to the virtual table information.

6 . The system for data replication of claim 3 , wherein the message producer comprises:
a message producer application programming interface executable to receive a virtual table package, the virtual table package including the virtual table information and the virtual table records; a message producer file service executable to store the virtual table information and the virtual table records to the persistent storage; and a producer service executable to write the references to the queue.

7 . The system for data replication of claim 1 , further comprising a gateway computer system, the gateway computer system comprising a gateway application programming interface.

8 . The system for data replication of claim 1 , wherein the message producer and the message consumer comprise instructions executable for:
determining if the physical table that corresponds to the virtual table exists in the cloud hosted database; and based on the determination that the physical table that corresponds to the virtual table does not exist in the cloud hosted database, creating the physical table in the cloud hosted database.

9 . The system for data replication of claim 1 , wherein the table schema does not match a pre-existing schema in the cloud hosted database.

10 . A system for data replication comprising:
a processor; a memory in communication with the processor, wherein the memory comprises:
a virtual table definition, the virtual table definition comprising:
a definition of a table schema that includes virtual table fields and data types of the virtual table fields; and
a mapping of the virtual table fields to target data types of a cloud hosted database; and

a data collection definition that defines data of interest from a data source;
a transformation engine that comprises instructions executable for:
creating a virtual table according to the virtual table definition, the virtual table comprising the virtual table fields; and
populating the virtual table fields with the data of interest according to the data collection definition; and

a cloud connector, the cloud connector comprising instructions executable for:
sending virtual table information and virtual table records to a cloud environment for replication of the data of interest in the cloud hosted database when the table schema does not match a pre-existing schema in the cloud hosted database or in the heterogeneous on-premises data sources, the virtual table information comprising:
the table schema; and
the mapping of the virtual table fields to the target data types;
creating the physical table in the cloud hosted database, wherein a structure of the physical table for replication is defined by the table schema and the mapping of the virtual table fields to the target data types received in the virtual table information.

11 . The system for data replication of claim 10 , wherein the data of interest comprises data from heterogeneous on-premises data sources.

12 . The system for data replication of claim 11 , wherein the table schema does not match a pre-existing schema in the cloud hosted database or in the heterogeneous on-premises data sources.

13 . The system for data replication of claim 10 , wherein the virtual table comprises a first virtual table field for holding a character string representing metadata of a respective file and a second virtual table field for holding the respective file as a binary string.

14 . The system for data replication of claim 10 , wherein the virtual table information is sent to the cloud environment as a text-based representation of the virtual table.

15 . A system for data replication comprising:
an on-premises system, the on-premises system comprising:
an on-premises system processor;
an on-premises system memory in communication with the on-premises system processor, wherein the on-premises system memory comprises:
a virtual table definition, the virtual table definition comprising:
a definition of a table schema that includes virtual table fields and data types of the virtual table fields; and
a mapping of the virtual table fields to target data types of a cloud hosted database; and

a data collection definition that defines data of interest from a data source;
a transformation engine that comprises instructions executable for:
creating a virtual table according to the virtual table definition, the virtual table comprising the virtual table fields; and
populating the virtual table fields with the data of interest according to the data collection definition; and

a cloud connector, the cloud connector comprising in

Assignees

• Open Text Sa Ulc

Inventors

• Lilko Alexander
• O'Hagan Paul

Classifications

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title

• G06F16/273 Primary

  Asynchronous replication or reconciliation · CPC title

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title