Methods, apparatus, and systems to aggregate partitioned computer database data

Patent metadata

Field	Value
Publication number	US-11138230-B2
Application number	US-201815935746-A
Country	US
Kind code	B2
Filing date	Mar 26, 2018
Priority date	Mar 26, 2018
Publication date	Oct 5, 2021
Grant date	Oct 5, 2021

Abstract

Methods, apparatus, systems and articles of manufacture are disclosed. An example partitioned computer database system includes a plurality of nodes, a data director to distribute a plurality of portions of database data across the plurality of nodes, queriers associated with respective ones of the plurality of nodes, the queriers to execute respective sub-queries of respective portions of the database data, and a coordinator to receive a request to query the database data, and merge results of the plurality of sub-queries to form a response to the request.

First claim

What is claimed is:

1. A partitioned database system, comprising:
a plurality of nodes;
a data director to randomly or pseudo-randomly distribute a plurality of portions of database data across the plurality of nodes without respect to content of the database data;
queriers associated with respective ones of the plurality of nodes, the queriers to execute respective ones of a plurality of sub-queries of respective portions of the database data; and
a coordinator to:
receive a request to query the database data, the request including at least one sharding key value provided as part of a conditional statement;
generate the plurality of sub-queries based on the at least one sharding key value;
instruct the queriers to execute the respective ones of the plurality of sub-queries and provide results of the execution of the respective ones of the plurality of sub-queries to the coordinator; and
merge the results of the plurality of sub-queries to form a response to the request.

2. The partitioned database system of claim 1 , wherein at least some of the plurality of nodes are to store their respective portions of the database data in a horizontally-arranged fact table.

3. The partitioned database system of claim 1 , wherein the queriers are to implement a distributed interface, the distributed interface to monitor a topology of storage devices associated with the nodes, and a real-time status of the partitioned database system.

4. The partitioned database system of claim 1 , wherein a first of the queriers is to perform the respective ones of the plurality of sub-queries without a shuffle of the respective portion of the database data.

5. The partitioned database system of claim 1 , wherein a first of the plurality of sub-queries is a linearly-scalable query.

6. The partitioned database system of claim 1 , wherein the coordinator is a first of the queriers, and the coordinator is to:
form the plurality of sub-queries based on the request; and
send the plurality of sub-queries to others of the queriers.

7. The partitioned database system of claim 6 , wherein the first of the queriers is to decompose the request to form the plurality of sub-queries.

8. The partitioned database system of claim 1 , wherein the request is received from a client application.

9. A method, comprising:
randomly or pseudo-randomly distributing respective portions of database data across a plurality of nodes without respect to content of the database data;
decomposing a request to query the database data, by executing an instruction with at least one processor, to form a plurality of sub-queries of respective portions of the database data based on at least one sharding key value, the request including the at least one sharding key value provided as part of a conditional statement;
instructing queriers to execute the sub-queries on respective ones of the nodes and provide results of the execution of the respective ones of the plurality of sub-queries; and
combining results of the plurality of sub-queries, by executing an instruction with at least one processor, to form a response to the request.

10. The method of claim 9 , wherein a first of the plurality of sub-queries does not shuffle the respective portion of the database data.

11. The method of claim 9 , wherein a first of the plurality of sub-queries is linearly scalable.

12. The method of claim 9 , wherein receiving of the query request and decomposing the query request to form the plurality of sub-queries is performed on a first node of the plurality of nodes, the first node to:
send the sub-queries to respective ones of the plurality of nodes;
receive the results of the sub-queries from the respective ones of the nodes; and
combine the results to form the response.

13. The method of claim 9 , wherein the combining of the results includes merging and reducing the results of the sub-queries.

14. A non-transitory computer-readable storage medium comprising instructions that, when executed, cause a machine to at least:
randomly or pseudo-randomly distributing respective portions of database data across a plurality of nodes of a partitioned database without respect to content of the database data;
decompose a request to query the database data to form a plurality of sub-queries of respective portions of the database data based on at least one sharding key value, the request including the at least one sharding key value provided as part of a conditional statement;
instruct queriers to execute the sub-queries to respective nodes on respective portions of the database data and provide results of the execution of the respective ones of the plurality of sub-queries; and
combine results of the plurality of sub-queries to form a response to the request.

15. The non-transitory computer-readable storage medium of claim 14 , wherein a first of the sub-queries is linearly scalable.

16. The non-transitory computer-readable storage medium of claim 14 , including further instructions that, when executed, cause the machine to combine the results by merging and reducing the results of the sub-queries.

17. The non-transitory computer-readable storage medium of claim 14 , wherein a first of the sub-queries does not shuffle the respective portion of the database data.

Assignees

• Mcafee Llc

Inventors

• Michelis Pablo A
• Stewart Brian H

Classifications

• G06F16/2471 Primary

  Distributed queries · CPC title

• G06F16/24534

  Query rewriting; Transformation · CPC title

• G06F16/278 Primary

  Data partitioning, e.g. horizontal or vertical partitioning · CPC title

• G06F16/24542

  Plan optimisation · CPC title

• G06F16/24554

  Unary operations; Data partitioning operations · CPC title