Processing queries based on rebuilding portions of virtual segments

Patent metadata

Field	Value
Publication number	US-11921725-B2
Application number	US-202217816070-A
Country	US
Kind code	B2
Filing date	Jul 29, 2022
Priority date	Jan 31, 2020
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

A processing system of a node is operable to determine at least one query for execution, and determine a set of segments that include a set of rows required to execute the at least one query. At least one virtual segment in the set of segments is identified, and at least two data portions in the at least one virtual segment are identified. A locally rebuilt set of rows for the at least two data portions is generated by utilizing a recovery scheme. Query resultant data is generated based on performing execution of the at least one query by utilizing the locally rebuilt set of rows.

First claim

What is claimed is:

1. A method for execution by a processing system of a node, comprising:
determining, by at least one processor of the processing system, at least one query for execution;
in response to the determining the at least one query:
determining, by the at least one processor of the processing system, a set of segments that include a set of rows required to execute the at least one query;
determining, by the at least one processor of the processing system, at least one virtual segment in the set of segments, wherein a first segment of the at least one virtual segment includes a plurality of sequential data portions, and wherein the first segment is included in a segment group generated by utilizing a redundancy coding scheme;
identifying, by the at least one processor of the processing system, at least two data portions in the at least one virtual segment, wherein the at least two data portions are a proper subset of the plurality of sequential data portions;
generating, by the at least one processor of the processing system, a locally rebuilt set of rows for the at least two data portions by utilizing a recovery scheme, wherein each sequential data portion of the plurality of sequential data portions is independently recoverable via the recovery scheme, and wherein the generating the locally rebuilt set of rows includes:
determining a plurality of segments in the segment group required to rebuild the first segment, wherein each segment of the plurality of segments includes a plurality of corresponding data portions;
identifying, for each segment of the plurality of segments, the at least two data portions of the corresponding data portions of the each segment of the plurality of segments required to rebuild the at least two data portions of the first segment;
retrieving the at least two data portions of the corresponding data portions of the each segment of the plurality of segments from a corresponding one of a set of memory drives, wherein the retrieving the at least two data portions of the corresponding data portions of the each segment of the plurality of segments includes generating a set of read requests to each corresponding one of the set of memory drives by batching a plurality of read requests to the each corresponding one of the set of memory drives, and wherein each read request of the plurality of read requests corresponds to one data portion of the at least two data portions of the corresponding each segment of the plurality of segments; and
utilizing the retrieved at least two data portions of the corresponding data portions of the each segment of the plurality of segments to rebuild the at least two data portions based on the redundancy coding scheme; and

generating, by the at least one processor of the processing system, query resultant data based on performing execution of the at least one query by utilizing the locally rebuilt set of rows.

2. The method of claim 1 , wherein the at least one virtual segment is included in at least one segment group generated by utilizing the redundancy coding scheme, and wherein generating the locally rebuilt set of rows includes:
determining the plurality of segments in the at least one segment group required to rebuild the at least two data portions;
retrieving at least a portion of each of the plurality of segments in the at least one segment group from a corresponding one of a set of memory drives; and
utilizing the retrieved at least the portion of the each segment of the plurality of segments in the segment group to rebuild the at least two data portions based on the redundancy coding scheme.

3. The method of claim 1 , wherein the first segment includes at least one other data portion, wherein the at least one other data portion and the at least two data portions are non-consecutive in the plurality of sequential data portions, further comprising:
generating a second locally rebuilt set of rows for the at least one other data portion by utilizing the recovery scheme, wherein the locally rebuilt set of rows is generated separately from the second locally rebuilt set of rows in response to the at least one other data portion and the at least two data portions being non-consecutive in the plurality of sequential data portions.

4. The method of claim 1 , wherein the at least one virtual segment is included in at least one segment group generated by utilizing the redundancy coding scheme, and wherein generating the segment group includes:
generating a plurality of data portion groupings by utilizing the redundancy coding scheme, wherein each data portion grouping includes one data portion from each segment of the plurality of segments in the segment group, wherein each data portion of a data portion grouping of the plurality of data portion groupings has a same placement in their corresponding plurality of sequential data portions of their corresponding data segment, and wherein each data portion is independently recoverable by utilizing other data portions in a same data portion grouping via the recovery scheme.

5. The method of claim 1 , wherein generating the locally rebuilt set of rows further includes: identifying subsets of the corresponding data portions of the each of the plurality of segments required to rebuild the at least two data portions; retrieving the subset of the corresponding data portions of the each of the plurality of segments from a corresponding one of a set of memory drives; and utilizing the retrieved subset of the corresponding data portions of the each of the plurality of segments to rebuild the at least two data portions based on the redundancy coding scheme.

6. The method of claim 1 , wherein identifying the at least two data portions in the at least one virtual segment includes identifying all consecutive data portions sequential to the at least two data portions that include ones of the set of rows required to execute the at least one query, wherein a first data portion is stored in sequence immediately prior to a first one of the at least two data portions, wherein a second data portion is stored in sequence immediately after a last one of the at least two data portions, and wherein the first data portion and the second data portion are not included in the at least two data portions based on determining the first data portion and the second data portion do not include ones of the set of rows required to execute the at least one query.

7. The method of claim 1 , wherein the at least one query includes a first query and a second query, wherein a first set of rows is required to execute the first query, wherein a second set of rows is required to execute the second query, wherein the at least two data portions include a first subset of the first set of rows, and wherein the at least two data portions further includes a second subset of the second set of rows.

8. The method of claim 7 , wherein the first subset of the first set of rows and the second subset of the second set of rows are mutually exclusive, wherein a first subset of the at least two data portions includes the first subset of the first set of rows, wherein a second subset of the at least two data portions includes the second subset of the second set of rows, and wherein the first subset of the at least two data portions and the second subset of the at least two data portions are mutually exclusive.

9. The method of claim 7 , wherein the first subset of the first set of rows and the second subset of the second set of rows have a non-zero intersection, and wherein at least one data portion of the at least two data portions includes rows of the first set of rows and the second set of rows.

10. The method of claim 1 , wherein the at least one virtual segment

Assignees

• Ocient Holdings LLC

Inventors

• Kondiles George
• Coombs Daniel

Classifications

• G06F16/24568 Primary

  Data stream processing; Continuous queries · CPC title

• G06F16/2456

  Join operations · CPC title

• G06Q50/04

  Manufacturing · CPC title

• G06F16/24532 Primary

  of parallel queries · CPC title

• Y02P90/30

  Computing systems specially adapted for manufacturing · CPC title