Point in time consistent materialization for object storage environments

Patent metadata

Field	Value
Publication number	US-11468059-B2
Application number	US-201816235098-A
Country	US
Kind code	B2
Filing date	Dec 28, 2018
Priority date	Oct 6, 2018
Publication date	Oct 11, 2022
Grant date	Oct 11, 2022

Abstract

A query that is frequently processed to access an object storage is identified. Results from the query returned from the object storage is transformed into a relational database format as a materialized view. When the query is submitted a subsequent time, updated results are managed from the materialized view, other materialized views, and/or the object storage when needed.

First claim

The invention claimed is:

1. A method, comprising:
processing a query submitted for an object storage, wherein the query is in a non-relational format;
transforming storage object values from storage objects in the object storage as results to the query, wherein the storage object values are transformed into a relational database format associated with tables, columns, and rows within a materialized view;
maintaining storage object identifiers and modification timestamps associated with the storage object values from the storage objects in the object storage and as column attributes within the materialized view for point-in-time consistent materialization for the object storage, wherein the object storage is accessed when the modification timestamps associated with the storage object values maintained as the column attributes within the materialized view are earlier than the modification timestamps associated with the storage object values in the object storage; and
providing the materialized view to satisfy the query when submitted again without accessing the object storage.

2. The method of claim 1 , wherein transforming further includes processing a Structured Query Language (SQL) SELECT Data Manipulation Language (DML) statement that defines a mapping between the storage object values to the relational database format.

3. The method of claim 1 wherein maintaining further includes, maintaining a filter representing text-formatted query conditions for the query with the materialized view.

4. The method of claim 3 , wherein maintaining further includes periodically submitting the filter for the query to the object storage and updating the materialized view with new modification timestamps and new storage object values.

5. The method of claim 4 , wherein updating further includes removing at least one storage object identifier and corresponding storage object values associated with the at least one storage object identifier.

6. The method of claim 4 , wherein updating further includes adding at least one storage object identifier and corresponding storage object values associated with the at least one storage object identifier.

7. The method of claim 1 , wherein providing further includes updating a portion of the storage object values with data values obtained from a different materialized view and a corresponding modification timestamp when a particular storage object identifier of the different materialized view matches a corresponding storage object identifier of the materialized view and has a more recent modification timestamp.

8. The method of claim 1 , wherein providing further includes updating a portion of the storage object values from the object storage based on a filter associated with a particular storage object identifier included in the materialized view.

9. The method of claim 1 , wherein providing further includes provide the materialized view to satisfy the query without accessing the object storage.

10. The method of claim 1 , wherein providing further includes updating the materialized view to satisfy the query by obtaining:
first storage object values from different materialized views and second storage object values from the object store.

11. The method of claim 10 further comprising, periodically refreshing the materialized view from the object storage based on a refresh policy.

12. A method comprising:
identifying a query that accesses an object storage, wherein the query is in a non-relational format;
transforming storage object values from storage objects in the object storage, wherein the storage object values are transformed into a relational database format associated with tables, columns, and rows as a materialized view providing results for the query;
maintaining storage object identifiers and modification timestamps associated for each storage object and as column attributes within the materialized view for point-in-time consistent materialization for the object storage, wherein the object storage is accessed when the modification timestamps associated with the storage object values maintained as the column attributes within the materialized view are earlier than the modification timestamps associated with the storage object values in the object storage; and
managing the materialized view to provide updated results for the query a next time the query is processed.

13. The method of claim 12 , wherein managing further includes using the storage object identifiers and the modification timestamps to determine where different portions of the materialized view are to be updated from.

14. The method of claim 13 , wherein using further includes obtaining some of the updated results from different materialized views.

15. The method of claim 14 , wherein obtaining further includes obtaining another portion of the updated results from the object storage based on filters associated with particular storage object identifiers in the materialized view.

16. The method of claim 12 further comprising, periodically refreshing the materialized view by obtaining new results from the object storage for the query based on a refresh policy.

17. The method of claim 12 further comprising, automatically obtaining new results from the object storage for the query when a determination is made that more than a predefined portion of the results are out of date with the object storage or other materialized views.

18. A system, comprising:
a database management system;
at least one hardware processor;
a non-transitory computer-readable storage medium having executable instructions representing a storage object materialized-view manager;
the storage object materialized-view manager configured to execute on the at least one hardware processor from the non-transitory computer-readable storage medium and to perform processing to:
i) identify a query that processes against an object storage, wherein the query is in a non-relational format;
ii) transform storage object values from storage objects in the object storage for the query, wherein the storage object values are transformed into a relational database format associated with tables, columns, and rows within a materialized view as results for the query;

iii) maintain storage object identifiers and modification timestamps associated with the storage object values from the storage objects in the object storage and as column attributes within the materialized view for point-in-time consistent materialization for the object storage, wherein the object storage is accessed when the modification timestamps associated with the storage object values maintained as the column attributes within the materialized view are earlier than the modification timestamps associated with the storage object values in the object storage; and
iv) manage updated results for the query when submitted a next time within the database management system from the materialized view, different materialized views associated with other queries, and the object storage.

19. The system of claim 18 , wherein the storage object materialized-view manager is further configured to perform processing to:
v) refresh the materialized view with new results obtained from the object storage.

Assignees

• Teradata Us Inc

Inventors

• Watzke Michael Warren
• Cohen Steven B
• Pederson Donald Raymond

Classifications

• G06F16/24539 Primary

  using cached or materialised query results · CPC title

• G06F16/284

  Relational databases · CPC title

• G06F16/2393 Primary

  Updating materialised views · CPC title