Managed materialized views created from heterogenous data sources

Patent metadata

Field	Value
Publication number	US-2022058187-A1
Application number	US-202117466347-A
Country	US
Kind code	A1
Filing date	Sep 3, 2021
Priority date	Nov 29, 2019
Publication date	Feb 24, 2022
Grant date	—

Abstract

Managed materialized views may be generated from across heterogeneous data sources. A request to create a materialized view may be received and performed by a materialized view management platform, which may obtain and generate the materialized view from different data sources and store the materialized view in a target system. Changes to the data sources may be obtained at the materialized view management platform and updates to the materialized view may be determined. The materialized view in the target system may be updated.

First claim

What is claimed is:

1 . A system, comprising:
at least one processor; and a memory, storing program instructions that when executed by the at least one processor, cause the at least one processor to implement a materialized view management platform, the materialized view management platform, configured to:
create a materialized view from one or more data sources that are specified as sources in a definition for a materialized view, wherein the materialized view management platform is capable of creating the materialized view from a plurality of different types of data sources, and wherein the materialized view is stored by the materialized view management platform in a target data store also specified in the definition of the materialized view;
obtain, for at least one of the the one or more data sources, a respective one or more changes made at the at least one data source;
determine a new version of the materialized view that satisfies the definition of the materialized view from the respective one or more changes obtained from the at least one data source; and
send one or more requests to update an existing version of the materialized view stored in the target data store to the new version of the materialized view via an interface for the target data store.

2 . The system of claim 1 , wherein to determine a new version of the materialized view that satisfies the definition of the materialized view from the respective one or more changes obtained from the at least one data source, the materialized view management platform is configured to generate a stream of updates to be performed to the materialized view as the changes are obtained from the one or more data sources.

3 . The system of claim 1 , wherein to obtain the respective one or more changes made at the at least one data source, the materialized view management platform is configured to enable a change data capture stream to be received from the at least one data source.

4 . The system of claim 1 , wherein the materialized view management platform is offered as part of a provider network, wherein at least one of the one or more data sources or the target is another service offered by the provider network.

5 . A method, comprising:
obtaining, by a materialized view management platform, a respective one or more changes made at one or more data sources that are specified as sources in a definition for a materialized view, wherein the materialized view management platform is capable of creating the materialized view from a plurality of different types of data sources, and wherein the materialized view is stored by the materialized view management platform in a target data store also specified in the definition of the materialized view; determining, by the materialized view management platform, a new version of the materialized view that satisfies the definition of the materialized view from the respective one or more changes obtained from the one or more data sources; and updating, by the materialized view management platform, an existing version of the materialized view stored in the target data store to the new version of the materialized view via an interface for the target data store.

6 . The method of claim 5 , further comprising:
receiving a request to create the materialized view that includes the definition of the materialized view; in response to receiving the request:
generating, according to the definition, a creation plan that includes instructions to:
query the one or more of data sources to obtain data for the materialized view;
create one or more intermediate tables to generate an internal version of the materialized view; and
generate the internal version of the materialized view from the one or more intermediate tables.

7 . The method of claim 6 , further comprising:
further in response to receiving the request to create the materialized view, generating and storing a maintenance plan for the materialized view according to the definition, wherein the maintenance plan includes operations to update the one or more intermediate tables and the internal version of the materialized view with changes received from the one or more data sources.

8 . The method of claim 5 , wherein determining, from the respective one or more changes obtained from the one or more data sources, the new version of the materialized view that satisfies the materialized view definition comprises executing the maintenance plan generated for the materialized view that includes the changes in intermediate tables and an internal version of the materialized view.

9 . The method of claim 8 , wherein updating the existing version of the materialized view stored in the target data store to the new version of the materialized view via the interface for the target data store comprises:
determining, from the internal version of the materialized view, one or more updates to perform to the materialized view in the target data store to match the internal version of the materialized view; and sending one or more requests via the interface to the target data store to perform the determined one or more updates to the materialized view in the target data store.

10 . The method of claim 5 , wherein obtaining the respective one or more changes made at the plurality of different types of data sources comprises:
for at least one of the different data sources, enabling a change data capture stream to be received from the at least one data source.

11 . The method of claim 5 , wherein obtaining the respective one or more changes made at the plurality of different types of data sources comprises:
for at least one of the different data sources:
sending a request to obtain those changes that occurred at the one data source after a previous request to obtain changes; and
receiving, from the one data source the respective one or more changes that were made at the one data source.

12 . The method of claim 5 , wherein obtaining the respective one or more changes made at the plurality of different types of data sources comprises querying respective delta tables for the different data sources to obtain the respective on or more changes made at the plurality of different types of data sources that were stored into the respective delta tables by respective connectors for the different types of data sources.

13 . The method of claim 5 , wherein the target data store is one of the one or more data sources.

14 . One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement a materialized view management platform that implements:
obtaining a respective one or more changes made at one or more data sources that are specified as sources in a definition for a materialized view, wherein the materialized view management platform is capable of creating the materialized view from a plurality of different types of data sources, and wherein the materialized view is stored by the materialized view management platform in a target data store also specified in the definition of the materialized view; determining a new version of the materialized view that satisfies the definition of the materialized view from the respective one or more changes obtained from the one or more data sources; and sending one or more requests to update an existing version of the materialized view stored in the target data store to the new version of the materialized view via an interface for the target data store.

15 . The one or more non-transitory, comp

Assignees

• Amazon Tech Inc

Inventors

• Deshpande Rohan
• Papakonstantinou Yannis
• Zaki Omer Ahmed
• Bharathy Karthik Gurumoorthy Subramanya
• Dugar Rajib
• Ummenthala Shivateja Reddy

Classifications

• G06F16/2445

  Data retrieval commands; View definitions · CPC title

• G06F16/219

  Managing data history or versioning (querying versioned data G06F16/2474; querying temporal data G06F16/2477) · CPC title

• G06F16/2393 Primary

  Updating materialised views · CPC title