Accessing a primary clustery key index structure during query execution

Patent metadata

Field	Value
Publication number	US-12591561-B2
Application number	US-202418946331-A
Country	US
Kind code	B2
Filing date	Nov 13, 2024
Priority date	Sep 27, 2022
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

A database system is operable to generate an IO pipeline that includes a primary cluster key pipeline element serially after a secondary index pipeline element. The IO pipeline is executed based on applying the secondary index pipeline element of the IO pipeline to generate row list output, and applying the primary cluster key pipeline element to determine a first set of row ranges, determine a second set of row ranges of row list output generated by the secondary index pipeline element, and generate, from the first set of row ranges and the second set of row ranges, a result set having a plurality of outputs, each indicating a cluster key and a row range for the cluster key.

First claim

What is claimed is:

1 . A database system comprises:
a plurality of computing device clusters, wherein a computing device cluster of the plurality of computing device clusters includes a plurality of computing devices, wherein a computing device of the plurality of computing devices includes a plurality of computing nodes, wherein a computing node of the plurality of computing nodes includes a plurality of processing core resources, wherein a set of processing core resources of the pluralities of processing core resources of the computing device cluster is operable to:
receive a query regarding a dataset, wherein the dataset includes a plurality of rows of columnar data, wherein:
the dataset is divided into a plurality of segments,
a first segment of the plurality of segments includes a first sub-set of rows of columnar data of the plurality of rows of columnar data,
the first sub-set of rows are organized in accordance with a primary cluster key,
a first secondary index structure is created for the first segment based on a secondary index, wherein the secondary index identifies rows of the dataset that include data values of interest;
the first segment and the first secondary index structure are stored in accordance with a long-term storage (LTS) protocol;

generate, in accordance with executing the query, a plurality of input/output (IO) pipelines, wherein each IO pipeline of the plurality of IO pipelines includes a secondary index element and a primary key element, and wherein a first IO pipeline of the plurality of IO pipelines is associated with the first processing core resource;
retrieve, in accordance with executing the query, the first segment by the secondary index element of the first IO pipeline in accordance with the first secondary index structure to produce first retrieved indexed rows of the first segment;
filter, in accordance with executing the query and based on a secondary filtering condition of the query, the first retrieved indexed rows of the first segment by the secondary index element of the first IO pipeline to produce first filtered indexed rows of the first segment; and
process, in accordance with executing the query, the first filtered indexed rows of the first segment by the primary key element of the first IO pipeline in accordance with the primary key and the LTS protocol to produce first raw, filtered, indexed rows of the first segment.

2 . The database system of claim 1 further comprises:
a second segment of the plurality of segments includes a second sub-set of rows of columnar data of the plurality of rows of columnar data;
the second sub-set of rows are organized in accordance with the primary cluster key;
a second secondary index structure is created for the second segment based on the secondary index;
the second segment and the second secondary index structure are stored in accordance with the LTS protocol;
a second IO pipeline of the plurality of IO pipelines is associated with the second processing core resource;
wherein a second IO pipeline of the plurality of IO pipelines is associated with the second processing core resource;
retrieve, in accordance with executing the query, the second segment by the second index element of the second IO pipeline in accordance with the second secondary index structure to produce second retrieved indexed rows of the second segment;
filter, in accordance with executing the query and based on the secondary filtering condition, the second retrieved indexed rows of the second segment by the secondary index element of the second IP pipeline to produce second filtered indexed rows of the second segment; and
process, in accordance with executing the query, the second filtered indexed rows of the second segment by the primary key element of the second IO pipeline in accordance with the primary key and the LTS protocol to produce second raw, filtered, indexed rows of the second segment.

3 . The database system of claim 2 further comprises:
the first segment and the first secondary index structure are stored in memory of the first processing core resource of the set of processing core resources; and
the second segment and the second secondary index structure are stored in memory of a first processing core resource of the set of processing core resources.

4 . The database system of claim 2 , wherein the processing the second filtered indexed rows of the second segment further comprises:
filter, in accordance with executing the query, the second filtered indexed rows of the second segment based on a primary filtering condition of the query to facilitate producing the second raw, filtered, indexed rows of the second segment.

5 . The database system of claim 2 , wherein the set of processing core resources are further operable to:
process, in accordance with executing the query, the first and second raw, filtered, indexed rows of the first and second segments to produce one or more partial query results.

6 . The database system of claim 1 , wherein the LTS protocol comprises one or more of:
dictionary compression; data compression; data deduplication; data integrity; and data error encoding.

7 . The database system of claim 1 , wherein the processing the first filtered indexed rows of the first segment further comprises:
filter, in accordance with executing the query, the first filtered indexed rows of the first segment based on a primary filtering condition of the query to facilitate producing the first raw, filtered, indexed rows of the second segment.

8 . A computer readable memory device comprises:
at least one memory section that stores operational instructions that, when executed by a set of processing core resources of a computing device cluster of a database system, causes the set of processing core resources to:
receive a query regarding a dataset, wherein the dataset includes a plurality of rows of columnar data, wherein:
the dataset is divided into a plurality of segments,
a first segment of the plurality of segments includes a first sub-set of rows of columnar data of the plurality of rows of columnar data,
the first sub-set of rows are organized in accordance with a primary cluster key,
a first secondary index structure is created for the first segment based on a secondary index, wherein the secondary index identifies rows of the dataset that include data values of interest;
the first segment and the first secondary index structure are stored in accordance with a long-term storage (LTS) protocol;
generate, in accordance with executing the query, a plurality of input/output (IO) pipelines, wherein each IO pipeline of the plurality of IO pipelines includes a secondary index element and a primary key element, and wherein a first IO pipeline of the plurality of IO pipelines is associated with the first processing core resource;

retrieve, in accordance with executing the query, the first segment by the secondary index element of the first IO pipeline in accordance with the first secondary index structure to produce first retrieved indexed rows of the first segment;
filter, in accordance with executing the query and based on a secondary filtering condition of the query, the first retrieved indexed rows of the first segment by the secondary index element of the first IO pipeline to produce first filtered indexed rows of the first segment; and
process, in accordance with executing the query, the first filter indexed rows of the first segment by the primary key element of the first IO pipeline in accordance with the primary key and the LTS protocol to produce first raw, filtered, indexed rows of the first segment, wherein the database system includes:
a plurality of computing device clusters, wherein the computing de

Assignees

• Ocient Holdings LLC

Inventors

• Wendel Iii Richard George
• Dhuse Greg R
• Farahani Hassan
• Ashbeck Matthew
• Veselova Anna
• Rabe Benjamin Daniel

Classifications

• G06F16/24542

  Plan optimisation · CPC title

• G06F16/24558

  Binary matching operations · CPC title

• G06F16/24561

  Intermediate data storage techniques for performance improvement · CPC title

• G06F16/24532

  of parallel queries · CPC title

• G06F16/2246 Primary

  Trees, e.g. B+trees · CPC title