Efficient pushdown of joins in a heterogeneous database system involving a large-scale low-power cluster

Patent metadata

Field	Value
Publication number	US-9613066-B2
Application number	US-201414496090-A
Country	US
Kind code	B2
Filing date	Sep 25, 2014
Priority date	Oct 4, 2012
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

A database management server manages a distributed data set at least partially stored in volatile memory of a plurality of nodes. The database management server stores transaction logs in persistent storage corresponding to one or more database transactions performed on the distributed data set. In response to a query, a join operation is performed with assistance from the cluster. A snapshot identifier identifies a particular database state that is transactionally consistent. Based on the transactional logs, updated data blocks updated after the snapshot identifier of the query are identified. The cluster is instructed to perform one or more distributed join operations on the distributed data set without processing data in the updated data blocks. The database management server performs a partial join evaluation on the updated data blocks.

First claim

What is claimed is:

1. A method comprising:
managing, by a database management server, a distributed data set at least partially stored in volatile memory of a plurality of nodes in a cluster of nodes;
storing in persistent storage, by the database management server, transaction logs corresponding to one or more database transactions performed on the distributed data set; and
performing a join operation of a query with assistance from the cluster by:
determining a snapshot identifier of the query, the snapshot identifier identifying a particular database state that is transactionally consistent;
identifying based on the transactional logs, by the database management server, updated data blocks updated after the snapshot identifier of the query;
instructing the cluster to perform one or more distributed join operations on the distributed data set without processing data in the updated data blocks; and
performing, by the database management server, a partial join evaluation on the updated data blocks.

2. The method of claim 1 , wherein performing the join operation further comprises causing affected data corresponding to one or more tables affected by the join operation to be written from any write buffers holding the affected data.

3. The method of claim 1 , wherein the database management server comprises one or more enhancements for performing partial join evaluations.

4. The method of claim 3 , wherein the one or more enhancements comprises a partial table scan configured to scan a subset comprising one or more data blocks.

5. The method of claim 1 , further comprising:
estimating a cost of performing the join operation with the assistance from the cluster;
estimating a cost of performing the join operation fully in the database management server; and
wherein performing said join operation with the assistance from the cluster is performed when the cost of performing the join operation with the assistance from the cluster is less than the cost of performing the join operation fully in the database management server.

6. The method of claim 5 , wherein the cost of performing the join operation fully in the database management server comprises the cost of sending one or more relations from the plurality of nodes to the database management server.

7. The method of claim 1 , wherein performing the join operation further comprises:
determining that a table to be joined is sufficiently small to be handled by nodes of the cluster; and
broadcasting the table to be joined to the plurality of nodes.

8. The method of claim 1 , further comprising partitioning, by the database management server, tables to be joined based on a join column, wherein individual partitions are processed by individual nodes of the plurality of nodes.

9. The method of claim 1 , wherein the join operation comprises a blocking operation, wherein performing the join operation further comprises:
integrating, by the database management server, blocking operation results;
partitioning the blocking operation results; and
sending the partitioned blocking operation results to the plurality of nodes.

10. A heterogeneous database system comprising:
a cluster comprising a plurality of processing units, each processing unit comprising at least one volatile memory unit and at least one processor configured to store at least a portion of a distributed data set in the at least one volatile memory unit;
a database management server comprising at least one persistent storage medium and at least one processor configured to store transaction logs in the at least one persistent storage medium, wherein the transaction logs correspond to one or more database transactions performed on the distributed data set;
wherein the database management server is further configured to perform a join operation with assistance from the cluster by:
determining a snapshot identifier corresponding to the join operation, the snapshot identifier identifying a particular database state that is transactionally consistent;
identifying, based on the transactional logs, updated data blocks updated after the snapshot identifier;
instructing the cluster to perform one or more distributed join operations on the distributed data set without processing data in the updated data blocks; and
performing, by the database management server, a partial join evaluation on the updated data blocks.

11. The heterogeneous database system of claim 10 , wherein performing the join operation further comprises causing affected data corresponding to one or more tables affected by the join operation to be written from any write buffers holding the affected data.

12. The heterogeneous database system of claim 10 , wherein the database management server comprises one or more enhancements for performing partial join evaluations.

13. The heterogeneous database system of claim 12 , wherein the one or more enhancements comprises a partial table scan configured to scan a subset comprising one or more data blocks.

14. The heterogeneous database system of claim 10 , wherein the database management server is further configured to:
estimate a cost of performing the join operation with the assistance from the cluster;
estimate a cost of performing the join operation fully in the database management server; and
wherein the database management server is configured to perform the join operation with the assistance from the cluster when the cost of performing the join operation with the assistance from the cluster is less than the cost of performing the join operation fully in the database management server.

15. The heterogeneous database system of claim 14 , wherein the cost of performing the join operation fully in the database management server comprises the cost of sending one or more relations from the plurality of processing units to the database management server.

16. The heterogeneous database system of claim 10 , wherein the database management server is further configured to perform the join operation with the assistance from the cluster by:
determining that a table to be joined is sufficiently small to handle by processing units of the cluster; and
broadcasting the table to be joined to the plurality of processing units.

17. The heterogeneous database system of claim 10 , wherein the database management server is further configured to partition tables to be joined based on a join column, wherein the processing units are further configured to process individual partitions.

18. The heterogeneous database system of claim 10 , wherein the join operation comprises a blocking operation, wherein database management server is further configured to:
integrate blocking operation results;
partition the blocking operation results; and
send the partitioned blocking operation results to the plurality of processing units.

19. A non-transitory computer readable medium storing instructions which, when executed by one or more processors of a database management system, causes:
managing a distributed data set at least partially stored in volatile memory of a plurality of nodes in a cluster of nodes;
storing in persistent storage, transaction logs corresponding to one or more database transactions performed on the distributed data set; and
performing a join operation of a query with assistance from the cluster by:
determining a snapshot identifier of the query, the snapshot identifier identifying a particular database state that is transactionally consistent;
identifying, based on the transactional logs, updated data blo

Assignees

• Oracle Int Company
• Oracle Int Corp

Inventors

• Idicula Sam
• Petride Sabina
• Agarwal Nipun
• Sedlar Eric

Classifications

• G06F17/30598

  Physics · mapped topic

• G06F17/30498

  Physics · mapped topic

• G06F17/30289 Primary

  Physics · mapped topic

• G06F16/2456 Primary

  Join operations · CPC title

• G06F16/21 Primary

  Design, administration or maintenance of databases · CPC title