System for list-based database replication

Patent metadata

Field	Value
Publication number	US-12475142-B2
Application number	US-202318494599-A
Country	US
Kind code	B2
Filing date	Oct 25, 2023
Priority date	May 29, 2020
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

A method of implementing sub-table replication starts with the processor detecting an update to an entitlements table. The processor performs filtering of a data table based on the update to the entitlements table. The data table including an entitlements column. The processor detects an update to the entitlements column and performs incremental replication of the data table by causing a version-based replication to be executed. Other embodiments are also described herein.

First claim

What is claimed is:

1 . A computer-implemented method of implementing sub-table replication comprising:
receiving, by a replication controller of a data platform, a first inclusion list indicating a first subset of rows from a data table of the data platform to replicate to a first provider account of the data platform; replicating, by the replication controller, the first subset of rows to the first provider account based on the first inclusion list; responsive to a provider user updating an entitlements table in the first provider account, performing operations comprising:
updating, by the replication controller, a virtual entitlements column stored in the data table indicating a change in provider accounts associated with a client identifier, the virtual entitlements column indicating a provider account associated with the first subset of rows of the data table;
identifying, by the replication controller, a second subset of rows in the data table associated with the client identifier by performing a filtering of the data table using file-level filtering of partitions of the data table based on metadata associated with the client identifier and the virtual entitlements column, the file-level filtering performed using the metadata without opening the partitions;
generating, by the replication controller, a second inclusion list comprising the second subset of rows associated with the client identifier based on the update to the virtual entitlements column; and
performing, by the replication controller, an incremental replication to replicate the second subset of rows to a second provider account based on the second inclusion list.

2 . The computer-implemented method of claim 1 , wherein the first inclusion list and the second inclusion list each further comprise data of a provider account and a region.

3 . The computer-implemented method of claim 1 , further comprising:
receiving the first inclusion list from a first client device associated with a provider user.

4 . The computer-implemented method of claim 3 , further comprising:
receiving the update to the entitlements table in the first provider account from the first client device.

5 . The computer-implemented method of claim 1 , wherein replicating the first subset of rows comprises:
identifying partitions containing the first subset of rows in the data table; and replicating the identified partitions.

6 . The computer-implemented method of claim 1 , wherein the data table is distributed across a plurality of storage devices.

7 . The computer-implemented method of claim 1 , wherein replicating the second subset of rows comprises transmitting the second subset of rows to the second provider account over a network.

8 . The computer-implemented method of claim 1 , wherein the incremental replication comprises a version-based replication.

9 . A data platform comprising:
at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the data platform to perform operations comprising: providing a first provider account; providing a second provider account; and providing a replication controller configured to perform first operations comprising:
receiving a first inclusion list indicating a first subset of rows from a data table to replicate to the first provider account;
replicating the first subset of rows to the first provider account based on the first inclusion list;
responsive to a provider user updating an entitlements table in the first provider account, performing second operations comprising:
updating a virtual entitlements column stored in the data table indicating a change in provider accounts associated with a client identifier, the virtual entitlements column indicating a provider account associated with the first subset of rows of the data table;
identifying a second subset of rows in the data table associated with the client identifier by performing a filtering of the data table using file-level filtering of partitions of the data table based on metadata associated with the client identifier and the virtual entitlements column, the file-level filtering performed using the metadata without opening the partitions;
generating a second inclusion list comprising the second subset of rows associated with the client identifier based on the update to the virtual entitlements column; and
performing an incremental replication to replicate the second subset of rows to a second provider account based on the second inclusion list.

10 . The data platform of claim 9 , wherein the first inclusion list and the second inclusion list each further comprise data of a provider account and a region.

11 . The data platform of claim 9 , wherein the operations further comprise:
receiving the first inclusion list from a first client device associated with a provider user.

12 . The data platform of claim 11 , wherein the operations further comprise:
receiving the update to the entitlements table in the first provider account from the first client device.

13 . The data platform of claim 9 , wherein replicating the first subset of rows comprises:
identifying partitions containing the first subset of rows in the data table; and replicating the identified partitions.

14 . The data platform of claim 9 , wherein the data table is distributed across a plurality of storage devices.

15 . The data platform of claim 9 , wherein replicating the second subset of rows comprises transmitting the second subset of rows to the second provider account over a network.

16 . The data platform of claim 9 , wherein the incremental replication comprises a version-based replication.

17 . A machine-storage medium comprising machine-readable instructions that, when executed by a machine, cause the machine to perform operations comprising:
receiving, by a replication controller of a data platform, a first inclusion list indicating a first subset of rows from a data table of the data platform to replicate to a first provider account of the data platform; replicating, by the replication controller, the first subset of rows to the first provider account based on the first inclusion list; responsive to a provider user updating an entitlements table in the first provider account, performing operations comprising:
updating, by the replication controller, a virtual entitlements column stored in the data table indicating a change in provider accounts associated with a client identifier, the virtual entitlements column indicating a provider account associated with the first subset of rows of the data table;
identifying, by the replication controller, a second subset of rows in the data table associated with the client identifier by performing a filtering of the data table using file-level filtering of partitions of the data table based on metadata associated with the client identifier and the virtual entitlements column, the file-level filtering performed using the metadata without opening the partitions;
generating, by the replication controller, a second inclusion list comprising the second subset of rows associated with the client identifier based on the update to the virtual entitlements column; and
performing, by the replication controller, an incremental replication to replicate the second subset of rows to a second provider account based on the second inclusion list.

18 . The machine-storage medium of claim 17 , wherein the first inclusion list and the second inclusion list each further comprise data of

Assignees

• Snowflake Inc

Inventors

• Chu Pui Kei Johnston
• Desai Shreyas Narendra
• Gil Echeverri German Alberto
• Krishnan Prasanna
• Mahesh Nithin
• Muralidhar Subramanian
• Robinson Eric
• Saini Sahaj

Classifications

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06Q30/0205 Primary

  based on location or geographical consideration · CPC title

• G06Q30/0185

  Product, service or business identity fraud · CPC title

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title

• G06F16/275 Primary

  Synchronous replication · CPC title