Maintaining data stream history for generating materialized views

Patent metadata

Field	Value
Publication number	US-11934389-B2
Application number	US-202217722224-A
Country	US
Kind code	B2
Filing date	Apr 15, 2022
Priority date	Nov 29, 2019
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Materialized views may be generated at a managed materialized view platform that accepts data streams as a source. When updating a materialized view with a data stream as a source, a stored portion of the stream is used to maintain a history of changes to be made. When a join operation needs to utilize the data stream source to determine an update to the materialized view, the data stream is preserved to complete the join and update the materialized view.

First claim

What is claimed is:

1. A system, comprising:
one or more processors; and
a memory, that stores program instructions that, when executed by the at least one processor, cause the one or more processors to implement a service of a provider network, configured to:
receive a request to create a materialized view stored at a target data store according to a definition that specifies a join key to identify matches between a first data source and a second data source, wherein application of the join key joins first data from the first data source that is a data stream with second data from the second data source;
create the materialized view at the target data store that joins the first data from the data stream with the second data from the second data source according to the definition of the materialized view that specifies the join key; and
maintain the materialized view to include further data obtained from the data stream joined with additional data obtained the second data source according to the definition of the materialized view that specifies the join key.

2. The system of claim 1 , wherein the second data source is a second data stream.

3. The system of claim 1 , wherein the second data source is a data store.

4. The system of claim 1 , wherein the materialized view is further created at a second target data store in addition to the target data store, wherein the target data store and the second target data store are selected according to a list of targets offered by the service of the provider network.

5. The system of claim 1 , wherein to maintain the materialized view to include the further data obtained from the data stream joined with the additional data obtained from the second data source, the service of the provider network is configured to access a stored portion of the data stream that includes the further data to determine a new version of the materialized view.

6. The system of claim 5 , wherein the service of the provider network is further configured to:
obtain the further data from the data stream;
append the further data as one or more entries in an append-only table maintained for the data stream, wherein the append-only table provides the stored portion of the data stream.

7. The system of claim 1 , wherein the target data store is another service offered by the provider network.

8. A method, comprising:
receiving, by a service of a provider network, a request to create a materialized view stored at a target data store according to a definition that specifies a join key to identify matches between a first data source and a second data source, wherein application of the join key joins first data from the first data source that is a data stream with second data from the second data source;
creating, by the service of the provider network, the materialized view at the target data store that joins the first data from the data stream with the second data from the second data source according to the definition of the materialized view that specifies the join key; and
maintaining, by the service of the provider network, the materialized view to include further data obtained from the data stream joined with additional data obtained the second data source according to the definition of the materialized view that specifies the join key.

9. The method of claim 8 , wherein the second data source is a second data stream.

10. The method of claim 8 , wherein the second data source is a data store.

11. The method of claim 8 , wherein the materialized view is further created at a second target data store in addition to the target data store, wherein the target data store and the second target data store are selected according to a list of targets offered by the service of the provider network.

12. The method of claim 8 , wherein maintaining the materialized view to include the further data obtained from the data stream joined with the additional data obtained from the second data source comprises accessing a stored portion of the data stream that includes the further data to determine a new version of the materialized view.

13. The method of claim 12 , further comprising:
obtaining, by the service of the provider network, the further data from the data stream;
appending, by the service of the provider network, the further data as one or more entries in an append-only table maintained for the data stream, wherein the append-only table provides the stored portion of the data stream.

14. The method of claim 8 , wherein the target data store is another service offered by the provider network.

15. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to implement:
receiving, by a service of a provider network, a request to create a materialized view stored at a target data store according to a definition that specifies a join key to identify matches between a first data source and a second data source, wherein application of the join key joins first data from the first data source that is a data stream with second data from the second data source;
creating, by the service of the provider network, the materialized view at the target data store that joins the first data from the data stream with the second data from the second data source according to the definition of the materialized view that specifies the join key; and
maintaining, by the service of the provider network, the materialized view to include further data obtained from the data stream joined with additional data obtained the second data source according to the definition of the materialized view that specifies the join key.

16. The one or more non-transitory computer-readable storage media of claim 15 , wherein the second data source is a second data stream.

17. The one or more non-transitory computer-readable storage media of claim 15 , wherein the second data source is a data store.

18. The one or more non-transitory computer-readable storage media of claim 15 , wherein the materialized view is further created at a second target data store in addition to the target data store, wherein the target data store and the second target data store are selected according to a list of targets offered by the service of the provider network.

19. The one or more non-transitory computer-readable storage media of claim 15 , wherein, in maintaining the materialized view to include the further data obtained from the data stream joined with the additional data obtained from the second data source, the program instructions cause the one or more computing devices to implement accessing a stored portion of the data stream that includes the further data to determine a new version of the materialized view.

20. The one or more non-transitory computer-readable storage media of claim 19 , storing further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement:
obtaining, by the service of the provider network, the further data from the data stream;
appending, by the service of the provider network, the further data as one or more entries in an append-only table maintained for the data stream, wherein the append-only table provides the stored portion of the data stream.

Assignees

• Amazon Tech Inc

Inventors

• Deshpande Rohan
• Papakonstantinou Yannis
• Zaki Omer Ahmed
• Bharathy Karthik Gurumoorthy Subramanya
• Dugar Rajib
• Ummenthala Shivateja Reddy

Classifications

• G06F16/2393 Primary

  Updating materialised views · CPC title

• G06F16/219

  Managing data history or versioning (querying versioned data G06F16/2474; querying temporal data G06F16/2477) · CPC title

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title

• G06F16/2456

  Join operations · CPC title

• G06F16/24568

  Data stream processing; Continuous queries · CPC title