Managing materialized views in eventually consistent distributed data stores

Patent metadata

Field	Value
Publication number	US-2019332698-A1
Application number	US-201815962896-A
Country	US
Kind code	A1
Filing date	Apr 25, 2018
Priority date	Apr 25, 2018
Publication date	Oct 31, 2019
Grant date	—

Abstract

The disclosed embodiments provide a system for managing a materialized view of data in a data store. During operation, the system obtains a first creation time of a first materialized view of the data in the data store. Next, the system verifies that the first materialized view is not affected by deletions applied to the data since the first creation time. The system then identifies one or more additions to the data that are not included in the first materialized view. Finally, the system applies the additions to the first materialized view to maintain an eventual consistency of the first materialized view with the data store.

First claim

What is claimed is:

1 . A method, comprising:
obtaining a first creation time of a first materialized view of data in a data store; verifying, by one or more computer system, that the first materialized view is not affected by deletions applied to the data since the first creation time; identifying, by the one or more computer systems, one or more additions to the data that are not included in the first materialized view; and applying, by the one or more computer systems, the one or more additions to the first materialized view to maintain an eventual consistency of the first materialized view with the data store.

2 . The method of claim 1 , further comprising:
when a second materialized view of the data is affected by deletions applied to the data since a second creation time of the second materialized view, recreating the second materialized view using one or more queries of the data store.

3 . The method of claim 2 , wherein the one or more queries comprise:
a first query of first-degree connections of a node in a graph database; and a second query of first-degree connections of a result of the first query.

4 . The method of claim 1 , further comprising:
when a second creation time of a second materialized view of the data is older than a delete history comprising the deletions, recreating the second materialized view using one or more queries of the data store.

5 . The method of claim 1 , wherein verifying that the first materialized view is not affected by deletions applied to the data comprises:
verifying that the deletions are not applied to any data in the first materialized view.

6 . The method of claim 1 , wherein identifying the one or more additions to the data comprises:
transmitting, to the data store, a query comprising a key associated with the first materialized view; and obtaining, from the data store, the one or more additions to the data in a response to the query.

7 . The method of claim 6 , wherein querying the data store for the one or more additions to the data comprises:
querying an eventually consistent data store for additions to the data since a change data capture (CDC) offset associated with creation of the first materialized view.

8 . The method of claim 6 , wherein querying the data store for the one or more additions to the data comprises:
querying an eventually consistent data store for additions to the data since a time represented by a time buffer applied to the first creation time.

9 . The method of claim 8 , wherein the time buffer comprises a maximum eventual consistency delay associated with maintaining the eventual consistency between the first materialized view and the eventually consistent data store.

10 . The method of claim 1 , further comprising:
receiving a trigger to update the first materialized view prior to verifying that the data in the first materialized view is not affected by the deletions applied to the data.

11 . The method of claim 10 , wherein the trigger is generated after at least one of:
a request for the data in the first materialized view is received; and a time to live (TTL) of the first materialized view has lapsed.

12 . The method of claim 1 , wherein the materialized view comprises a select-project-join (SPJ) view.

13 . The method of claim 1 , wherein the data store comprises a graph database.

14 . A method, comprising:
obtaining a first materialized view of data in an eventually consistent data store; determining, by a computer system, a start time for performing incremental maintenance of the first materialized view, wherein the start time is based on an eventual consistency attribute associated with the first materialized view; querying the eventually consistent data store for one or more additions to the data since the start time; and applying, by the computer system, the one or more additions to the first materialized view to maintain an eventual consistency of the first materialized view with the eventually consistent data store.

15 . The method of claim 14 , further comprising:
when a second materialized view of the data is affected by deletions applied to the eventually consistent data store since a second creation time of the second materialized view, recreating the second materialized view from one or more queries of the eventually consistent data store.

16 . The method of claim 14 , wherein determining the start time associated with incremental maintenance of the first materialized view comprises:
setting the start time to a time represented by a change data capture (CDC) offset associated with creation of the first materialized view.

17 . The method of claim 14 , wherein determining the start time associated with incremental maintenance of the first materialized view comprises:
applying a time buffer to a creation time of the first materialized view to obtain the start time.

18 . The method of claim 17 , wherein the time buffer is greater than a maximum eventual consistency delay associated with maintaining eventual consistency between the first materialized view and the eventually consistent data store.

19 . The method of claim 14 , wherein the materialized view comprises a select-project-join (SPJ) view of the eventually consistent data store.

20 . A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:
obtaining a first creation time of a first materialized view of data in a data store; verifying that the first materialized view is not affected by deletions applied to the data since the first creation time; identifying one or more additions to the data that are not included in the first materialized view; and applying the one or more additions to the first materialized view to maintain an eventual consistency of the first materialized view with the data store.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Cho Sungju
• Averbukh Roman A
• Zhang Yanwei
• Carter Andrew J
• Jan Jane Alam

Classifications

• G06F16/2393

  Updating materialised views · CPC title

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title

• G06F16/2456

  Join operations · CPC title

• G06F16/9024

  Graphs; Linked lists (G06F16/9027 takes precedence) · CPC title

• G06F16/24565

  Triggers; Constraints · CPC title