Distributing large amounts of global metadata using object files

Patent metadata

Field	Value
Publication number	US-12591591-B2
Application number	US-202418897473-A
Country	US
Kind code	B2
Filing date	Sep 26, 2024
Priority date	Sep 30, 2022
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

A data dictionary generation system automatically populates and updates a data dictionary for listings offering shared data. The data listing distribution component distributes the data dictionaries to various remote deployments in a data exchange by using a global messaging framework and replication method. For example, the data listing distribution component replicates a data dictionary generated for the listing and its shared data from a source deployment to one or more destination deployments associated with various geographic regions. The data listing distribution component distributes the listing to the various remote deployments to allow for the listing, including its shared data and data dictionary, to be accessed by users within the geographic region associated with the remote deployment.

First claim

What is claimed is:

1 . A method comprising:
accessing, at a source deployment, a data dictionary generated for a listing object, the data dictionary describing shared data offered by the listing object; writing, by at least one hardware processor, the data dictionary to a global cloud object storage bucket associated with a destination deployment; selectively controlling visibility of the shared data and the data dictionary to consumers at the destination deployment based on a comparison of a deployment location identifier with region information corresponding to the listing object; overwriting a manifest file associated with the listing object to point to the shared data and the data dictionary replicated at the destination deployment; implementing a delay period between overwriting the manifest file and deleting a previous version of the shared data and the data dictionary; and after the delay period, deleting the previous version of the shared data and the data dictionary replicated at the destination deployment.

2 . The method of claim 1 , further comprising:
based on writing the data dictionary, transmitting a notification to the destination deployment, the notification causing the destination deployment to replicate shared data and the data dictionary at the destination deployment.

3 . The method of claim 1 , wherein the destination deployment uses the shared data and the data dictionary to provide the shared data offered by the listing object at the destination deployment and a description of the shared data that includes the data dictionary, further comprising:
receiving, at the source deployment, an approval decision from an exchange administrator for a publishing request; updating a status of a request field and a reason for rejection field in an exchange admin slice of a publishing request data processing object; transmitting a fulfillment message from the source deployment to the destination deployment to update a data provider slice of a listing approval request data processing object; and updating, via the fulfillment message, the status of a request field and the reason for rejection field in the data provider slice of the listing approval request data processing object at the destination deployment.

4 . The method of claim 3 , the shared data and the data dictionary replaces a previous version of the shared data and the data dictionary at the destination deployment.

5 . The method of claim 4 , wherein the destination deployment replaces the previous version of the shared data and the data dictionary by overwriting a manifest file associated with the listing object to point to the shared data and the data dictionary and deleting the previous version of the shared data and the data dictionary at the destination deployment.

6 . The method of claim 1 , further comprising:
determining that the deployment location identifier of the destination deployment is excluded from a visibility list; and in response to determining that the deployment location identifier is excluded from the visibility list, preventing the consumer associated with the destination deployment from viewing the shared data and the data dictionary of the listing object.

7 . The method of claim 1 , further comprising:
generating the data dictionary for the listing object; and writing the data dictionary to a local cloud object storage bucket of the source deployment, wherein the data dictionary is accessed from the local cloud object storage bucket.

8 . The method of claim 1 , further comprising:
receiving, at the destination deployment, a request from a consumer for access to the listing object, the request generated using listing metadata that indicates who a data provider is and where the data provide is from and an origin of the source deployment to determine where to send the request; transmitting a global message from the destination deployment to the source deployment, the global message requesting access to the shared data offered by the listing object; and creating an access request data processing object at the source deployment for managing approval or denial of the request for access to the listing object.

9 . The method of claim 1 , further comprising:
accessing an updated data dictionary generated for the listing object, the updated data dictionary describing updated shared data offered by the listing object; identifying the destination deployment to which the listing object is to be replicated; and writing the updated data dictionary and the updated shared data offered by the listing object to the global cloud object storage bucket associated with the destination deployment.

10 . The method of claim 1 , further comprising:
accessing a region visibility associated with the listing object, the region visibility identifying remote deployment at which the listing object is authorized to offer the shared data.

11 . A system comprising:
one or more computer processors; and one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising: accessing, at a source deployment, a data dictionary generated for a listing object, the data dictionary describing shared data offered by the listing object; writing, by at least one hardware processor, the data dictionary to a global cloud object storage bucket associated with a destination deployment; selectively controlling visibility of the shared data and the data dictionary to consumers at the destination deployment based on a comparison of a deployment location identifier with region information corresponding to the listing object; overwriting a manifest file associated with the listing object to point to the shared data and the data dictionary replicated at the destination deployment; implementing a delay period between overwriting the manifest file and deleting a previous version of the shared data and the data dictionary; and after the delay period, deleting the previous version of the shared data and the data dictionary replicated at the destination deployment.

12 . The system of claim 11 , wherein the selectively controlling of the visibility is performed by the destination deployment after the shared data and the data dictionary are replicated to the destination deployment, wherein visibility of the shared data and the data dictionary is prevented based on the comparison of the deployment location identifier with the region information corresponding to the listing object.

13 . The system of claim 11 , wherein the destination deployment uses the shared data and the data dictionary to provide the shared data offered by the listing object at the destination deployment and a description of the shared data that includes the data dictionary.

14 . The system of claim 13 , the shared data and the data dictionary replaces a previous version of the shared data and the data dictionary at the destination deployment.

15 . The system of claim 14 , wherein the destination deployment replaces the previous version of the shared data and the data dictionary replicated by overwriting a manifest file associated with the listing object to point to the shared data and the data dictionary and deleting the previous version of the shared data and the data dictionary replicated at the destination deployment.

16 . The system of claim 11 , wherein the operations comprise:
accessing a visibility list to retrieve a list of deployment identifiers for which the listing is available to be visible; determining that the depl

Assignees

• Snowflake Inc

Inventors

• Arikatla Durga Mahesh
• Muralidhar Subramanian
• Paladugu Vishnu Dutt
• Pulatova Shakhina
• Wu Di
• Xu Ziqi

Classifications

• G06Q30/0601

  Electronic shopping [e-shopping] · CPC title

• G06Q30/0206 Primary

  Price or cost determination based on market factors · CPC title

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title