Parallelized in-place radix sorting

Patent metadata

Field	Value
Publication number	US-10831738-B2
Application number	US-201715852045-A
Country	US
Kind code	B2
Filing date	Dec 22, 2017
Priority date	Jan 29, 2014
Publication date	Nov 10, 2020
Grant date	Nov 10, 2020

Abstract

Apparatuses and Methods for sorting a data set. A data storage is divided into a plurality of buckets that is each associated with a respective key value. A plurality of stripes is identified in each bucket. A plurality of data stripe sets is defined that has one stripe within each respective bucket. A first and a second in-place partial bucket radix sort are performed on data items contained within the first and second data stripe sets, respectively, using an initial radix. Incorrectly sorted data items in the first bucket are grouped by a first processor and incorrectly sorted data items in the second bucket are grouped by a second processor into a respective incorrect data item group within each bucket. A radix sort is then performed using the initial radix on the items within the respective incorrect data item group. A first level sorted output is produced.

First claim

What is claimed is:

1. A method comprising:
dividing a data storage into a plurality of buckets comprising at least a first bucket and a second bucket, each bucket being associated with a respective key value, the data storage containing a plurality of data items;
identifying, within each bucket of the plurality of buckets, a respective plurality of stripes;
defining a plurality of data stripe sets, each data stripe set in the plurality of data stripe sets comprising one respective stripe within each respective bucket of the plurality of buckets, the plurality of data stripe sets comprising at least a first data stripe set and a second data stripe set that is separate from the first data stripe set;
performing a first in-place partial bucket radix sort using an initial radix value on data items contained within the first data stripe set within the plurality of data stripe sets;
performing a second in-place partial bucket radix sort using the initial radix value on data items contained within the second data stripe set;
grouping, after performing the first in-place partial bucket radix sort and the second in-place partial bucket radix sort, incorrectly sorted data items in each bucket into a respective incorrect data item group within each bucket, the incorrectly sorted data items comprising data items within the first bucket but having a respective radix value associated with a different bucket,
the grouping comprising grouping incorrectly sorted data items in the first bucket with a first processor and grouping incorrectly sorted data items in the second bucket with a second processor that is different from the first processor while grouping incorrectly sorted data items in the first bucket with the first processor;
performing a radix sort using the initial radix value on items within the respective incorrect data item group; and
producing a first level sorted output comprising the plurality of data items within the data storage sorted according to the initial radix value.

2. The method of claim 1 , wherein the second in-place partial bucket radix sort is performed with the second processor.

3. The method of claim 1 ,
wherein the dividing comprises defining a plurality of data pointers into the data storage, each data pointer indicating a separation between two adjacent buckets, and
wherein the defining the plurality of data stripe sets comprises:
defining a respective stripe head pointer for each respective stripe, each respective stripe head pointer indicating a first element of each respective stripe; and
defining a respective stripe tail pointer for each respective stripe, each respective stripe tail pointer indicating a last element of each respective stripe.

4. The method of claim 1 , wherein each stripe in a particular data stripe set contains an equal number of data items.

5. The method of claim 1 , wherein at least one stripe within a data stripe set contains a first number of data elements unequal to a second number of data elements within stripes of the data stripe set that are other than the at least one stripe.

6. The method of claim 5 , wherein the first number of data elements and the second number of data elements are based upon value distribution characteristics of the plurality of data items.

7. The method of claim 1 , further comprising:
dividing, subsequent to performing the radix sort using the initial radix value on items within the respective incorrect data item group, at least one bucket within the plurality of buckets into a plurality of second level buckets, each second level bucket being associated with a respective second level key value;
identifying, within at least one second level bucket, a plurality of respective second level stripes;
defining at least one second level data stripe set comprising one respective second level stripe within each second level bucket of a respective at least one bucket; and
performing, with the first processor, a partial bucket radix sort on the first level sorted output using a second level radix value on data items contained within the at least one second level data stripe set.

8. The method of claim 7 , further comprising selecting the at least one second level bucket based upon a number of data elements in the at least one second level bucket relative to a number of data elements in other second level buckets within the plurality of buckets.

9. A data set sorting apparatus, comprising:
a multiple processor computing apparatus comprising a first processor and a second processor;
a memory coupled to the multiple processor computing apparatus;
a data sorting processor, coupled to the multiple processor computing apparatus and the memory, the data sorting processor configured to:
divide a data storage into a plurality of buckets comprising at least a first bucket and a second bucket, each bucket being associated with a respective key value, the data storage containing a plurality of data items;
identify, within each bucket of the plurality of buckets, a respective plurality of stripes;
define a plurality of data stripe sets, each data stripe set in the plurality of data stripe sets comprising one respective stripe within each respective bucket of the plurality of buckets, the plurality of data stripe sets comprising at least a first data stripe set and a second data stripe set that is separate from the first data stripe set;
perform a first in-place partial bucket radix sort using an initial radix value on data items contained within the first data stripe set;
perform a second in-place partial bucket radix sort using the initial radix value on data items contained within the second data stripe set;
group, after performing the first in-place partial bucket radix sort and the second in-place partial bucket radix sort, incorrectly sorted data items in each bucket into a respective incorrect data item group within each bucket, the incorrectly sorted data items comprising data items within the first bucket but having a respective radix value associated with a different bucket,
the data sorting processor is configured to group by at least grouping incorrectly sorted data items in the first bucket with a first processor and grouping incorrectly sorted data items in the second bucket with a second processor that is different from the first processor while grouping incorrectly sorted data items in the first bucket with the first processor;
perform a radix sort using the initial radix value on items within the respective incorrect data item group; and
produce a first level sorted output comprising the plurality of data items within the data storage sorted according to the initial radix value.

10. The apparatus of claim 9 , wherein the second in-place partial bucket radix sort is performed with the second processor.

11. The apparatus of claim 9 , wherein the data sorting processor is configured to:
divide the data storage by at least defining a plurality of data pointers into the data storage, each data pointer indicating a separation between two adjacent buckets, and
defining the plurality of data stripe sets by at least:
defining a respective stripe head pointer for each respective stripe, each respective stripe head pointer indicating a first element of each respective stripe; and
defining a respective stripe tail pointer for each respective stripe, each respective stripe tail pointer indicating a last element of each respective stripe.

12. The apparatus of claim 9 , wherein each stripe in a particular data stripe set contains an equal number of data items.

13. The apparatus of claim 9 , wherein at least one stripe within a data stripe set contains a first number o

Assignees

• IBM

Inventors

• Bordawekar Rajesh
• Brand Daniel
• Cho Minsik
• Finkler Ulrich
• Puri Ruchir

Classifications

• G06F16/23 Primary

  Updating · CPC title

• G06F16/2237 Primary

  Vectors, bitmaps or matrices · CPC title

• G06F16/285 Primary

  Clustering or classification · CPC title

• G06F16/24532

  of parallel queries · CPC title