Techniques for accessing a parallel database system via external programs using vertical and/or horizontal partitioning

Patent metadata

Field	Value
Publication number	US-9336270-B2
Application number	US-201414218160-A
Country	US
Kind code	B2
Filing date	Mar 18, 2014
Priority date	Dec 29, 2011
Publication date	May 10, 2016
Grant date	May 10, 2016

Abstract

Techniques for accessing a parallel database system via an external program using vertical and/or horizontal partitioning are provided. An external program to a database management system (DBMS) configures external mappers to process a specific portion of query results on specific access module processors of the DBMS that are to house query results. The query is submitted by the external program to the DBMS and the DBMS is directed to organize the query results in a vertical or horizontal manner. Each external mapper accesses its portion of the query results for processing in parallel on its designated AMP or set of AMPS to process the query results.

First claim

The invention claimed is:

1. A method, comprising:
configuring, by a processor, mappers for handling results from a query;
configuring each mapper to interact with a unique portion of the results from remaining ones of the mappers, wherein configuring each mapper further includes configuring, by the processor, each node of a plurality of nodes associated with the mappers for receiving all of the results for the query for ensuring that each mapper has access to all of the results on that mapper's node;
submitting the query for processing; and
passing a query identifier for the query to each of the mappers for handling that mapper's unique portion of the results returned for the processed query.

2. The method of claim 1 , wherein configuring the mappers further includes identifying sets of the mappers by interacting with access module processors for a Distributed Database Management System (DBMS), each access module processor having a unique set of the mappers.

3. The method of claim 2 further comprising, removing some of the mappers from a particular set for handling the results when a total number of the mappers is greater than a total number of the access module processors.

4. The method of claim 1 , wherein submitting the query further includes sending the query to a Distributed Database Management System (DBMS) for processing.

5. The method of claim 4 , wherein sending further includes instructing the DBMS to store the results in a Partitioned Primary Index (PPI).

6. The method of claim 5 , wherein instructing further includes resolving a total number of partitions for the PPI based on a total number of the mappers.

7. The method of claim 4 , wherein sending further includes reformatting the query before sending to the DBMS to include query processing recognized by the DBMS that results in the results being stored in a Partitioned Primary Index (PPI).

8. The method of claim 7 , wherein reformatting further includes associating the PPI with horizontal partitions of the DBMS.

9. The method of claim 8 further comprising configuring access module processors of the DBMS, each access module processor configured to handle one of the horizontal partitions, and each access module having a unique set of the mappers.

10. A method, comprising:
acquiring, by a processor, a total number of access module processors (AMPs) for a Distributed Database Management System (DBMS) and a total number of mappers;
assigning sets of the mappers to the AMPs;
receiving a query directed to the DBMS;
configuring each set of the mappers to handle a portion of results from the query, wherein configuring each set of mappers further includes configuring, by the processor, each node of a plurality of nodes associated with the sets of mappers for receiving all of the results for the query for ensuring that each set of mappers has access to all of the results on that set's node; and
submitting the query to the DBMS for processing.

11. The method of claim 10 , wherein acquiring further includes removing some of the mappers when the total number of the AMPs exceed the total number of the mappers.

12. The method of claim 10 , wherein assigning further includes assign each unique mapper to a unique one of the AMPs when the total number of the mappers is equal to the total number of the AMPs.

13. The method of claim 10 , wherein assigning further includes assigning at least one mapper to two or more of the AMPs.

14. The method of claim 10 , wherein configuring further includes assigning each set of mappers one or more unique rows in a table for the results that is returned by the DBMS in processing the query.

15. The method of claim 10 , wherein configuring further includes directing each mapper to connect in parallel to one another with the DBMS to simultaneously receive the results returned by the DBMS in processing the query.

16. The method of claim 10 , wherein configuring further includes providing each portion of the results as a unique range for the results.

17. The method of claim 10 , wherein submitting further includes instructing the DBMS to leave the results unsorted when submitting the query to the DBMS.

18. A Distributed Database Management System (DBMS), comprising:
network nodes distributed and in communication over a network;
at least one of the network nodes configured with an integrator module that executes on that at least one of the network nodes, and the integrator module configured to: i) execute on the at least one of the network nodes, ii) configure and assign mappers for handling results from a query submitted to the DBMS, and each mapper configured to handle a unique portion of the results, and configure each network node to receive all of the results for the query to ensure that each mapper has access to all of the results from that mapper's network node, and iii) submit the query to the DBMS for processing.

19. The DBMS of claim 18 , wherein the integrator module is further configured to, in ii), assign each portion as a unique vertical partitioning of the results to each of the mappers.

20. The DBMS of claim 18 , wherein the integrator module is further configured to, in ii), assign each portion as a unique horizontal partitioning of the results to each of the mappers.

Assignees

• Teradata Us Inc

Inventors

• Xu Yu
• Kostamaa Olli Pekka

Classifications

• G06F17/30584

  Physics · mapped topic

• G06F17/30424 Primary

  Physics · mapped topic

• G06F16/278 Primary

  Data partitioning, e.g. horizontal or vertical partitioning · CPC title

• G06F16/245 Primary

  Query processing · CPC title