Dynamically adjusting performance of materialized view maintenance

Patent metadata

Field	Value
Publication number	US-11899659-B2
Application number	US-201916699530-A
Country	US
Kind code	B2
Filing date	Nov 29, 2019
Priority date	Nov 29, 2019
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

The maintenance of a materialized view created and managed by a materialized view management platform may be dynamically adjusted. The performance of updates to a materialized view are monitored to trigger different events to make adjustments to the performance of subsequent updates. The materialized view management platform can adequately scale to handle changes to data sources as inputs to the maintenance of the materialized view, as well as make other adjustments handle various changes in the capability of the target data store to receive updates to the materialized view (e.g., target data store outages).

First claim

What is claimed is:

1. A system, comprising:
at least one processor; and
a memory, that stores program instructions that when executed by the at least one processor cause the at least one processor to implement a materialized view management platform, configured to:
create a materialized view from one or more of a plurality of different data sources implemented by different respective storage systems that the materialized view management platform is capable of accessing to store the materialized view in a target data store implemented by a storage system according to a definition for the materialized view, wherein to create the materialized view, the materialized view management platform allocates one or more computing resources of the materialized view management platform to maintain the materialized view, wherein the one or more computing resources of the materialized view management platform are implemented separately from the different respective storage systems implementing the plurality of different data sources and the storage system implementing the target data store;
monitor respective performance of operations performed by the allocated computing resources of the materialized view management platform to:
obtain changes made to the one or more data sources via respective interfaces of the one or more data sources;
determine updates to be made to the materialized view according to respective operation assignments, wherein the respective operation assignments select between local performance at the allocated computing resources or remote performance at one of the one or more data sources, wherein the respective operation assignments are determined based, at least in part, on computation capabilities of the one or more data sources; and
perform target-specific update translation that performs at least one data type conversion or operation conversion in addition to generating request parameters according to an interface of the target data store to send one or more requests with the generated request parameters to make the determined updates to the materialized view stored in the target data store, wherein the interface of the target data store supports one or more different features than the respective interfaces of the one or more data sources;

detect an event to adjust the performance of updating the materialized view based on the monitoring;
in response to the detection of the event:
scale the allocated computing resources to adjust the performance of updating the materialized view based on the detected event, wherein the scaling of the allocated computing resources causes an adjustment to subsequent operations to perform at least one of:
obtaining further changes for the materialized view from the different data sources;
determining further updates to make to the materialized view according to further respective operation assignments, wherein the further respective operation assignments select between local performance at the scaled computing resources or remote performance at one of the one or more data sources, wherein the further respective operation assignments are determined based, at least in part, on the computation capabilities of the one or more data sources; or
performing target-specific update translation to generate further request parameters according to the interface of the target data store to send one or more further requests with the generated request parameters to make the further updates to the materialized view in the target data store.

2. The system of claim 1 , wherein the detected event to adjust the performance of updating the materialized view indicates that the target data store is unavailable, and wherein the adjustment to the subsequent operations making the updates to the materialized view in the target data store comprises instructing a target connector for the target data store to buffer received updates to the materialized view.

3. The system of claim 1 , wherein the adjustment to the subsequent operations to make the further updates to the materialized view in the target data store comprises increasing or decreasing an update rate for sending the one or more further requests to the materialized view in the target data store.

4. The system of claim 1 , wherein the materialized view management platform is offered as part of a provider network and wherein at least one of the different data sources is another service offered by the provider network.

5. A method, comprising:
monitoring, by one or more computing devices implementing a materialized view management platform, respective performance of one or more computing resources allocated to maintain a materialized view when the materialized view is created by the materialized view management platform to:
obtain changes made via respective interfaces to one or more of a plurality of different data sources used to create a materialized view stored in a target data store, wherein the one or more computing resources of the materialized view management platform are implemented separately from different respective storage systems implementing the plurality of different data sources and a storage system implementing the target data store;
determine updates to be made to the materialized view according to respective operation assignments, wherein the respective operation assignments select between local performance at the allocated computing resources or remote performance at one of the one or more data sources, wherein the respective operation assignments are determined based, at least in part, on computation capabilities of the one or more data sources; and
performing target-specific update translation that performs at least one data type conversion or operation conversion in addition to generating request parameters according to an interface of the target data store to send one or more requests with the generated request parameters to make the determined updates to the materialized view stored in the target data store, wherein the interface of the target data store supports one or more different features than the respective interfaces of the one or more data sources;

detecting an event to adjust the performance of updating the materialized view based on the monitoring;
in response to detecting the event:
scaling the allocated computing resources to adjust the performance of updating the materialized view based on the detected event, wherein the scaling of the allocated computing resources causes an adjustment to subsequent operations to perform at least one of:
obtaining further changes for the materialized view from the different data sources;
determining further updates to make to the materialized view according to further respective operation assignments, wherein the further respective operation assignments select between local performance at the scaled computing resources or remote performance at one of the one or more data sources, wherein the further respective operation assignments are determined based, at least in part, on the computation capabilities of the one or more data sources; or
performing target-specific update translation to generate further request parameters according to the interface of the target data store to send one or more further requests with the generated request parameters to make the further updates to the materialized view in the target data store.

6. The method of claim 5 , wherein the adjustment to the subsequent operations obtaining the changes for the materialized view from the different data sources comprises increasing or decreasing a change capture rate for one or more of the different data sources.

7. The method of claim 5 , wherein the adjustment to the subsequent operations to make the further updates to the materialized view in the ta

Assignees

• Amazon Tech Inc

Inventors

• Deshpande Rohan
• Papakonstantinou Yannis
• Zaki Omer Ahmed
• Bharathy Karthik Gurumoorthy Subramanya
• Dugar Rajib
• Ummenthala Shivateja Reddy

Classifications

• G06F16/2445 Primary

  Data retrieval commands; View definitions · CPC title

• G06F11/3495 Primary

  for systems · CPC title

• G06F16/21

  Design, administration or maintenance of databases · CPC title

• G06F16/2393

  Updating materialised views · CPC title

• G06F2201/80

  Database-specific techniques · CPC title