Extensible data skipping

Patent metadata

Field	Value
Publication number	US-2020311088-A1
Application number	US-201916371019-A
Country	US
Kind code	A1
Filing date	Mar 31, 2019
Priority date	Mar 31, 2019
Publication date	Oct 1, 2020
Grant date	—

Abstract

Embodiments of the present systems and methods may provide techniques for data skipping that may be user defined and extensible. For example, a method may comprise receiving a query, at a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, modifying, at the computer system, the received query to utilize at least one data skipping index to evaluate at least one criterion of the query, wherein the at least one data skipping index comprises an index on at least one attribute of data that excludes index information for at least a portion of those data items that do not meet the at least one criterion, and wherein the at least one data skipping index is generated based on information received from an application programming interface, and evaluating, at the computer system, the query.

First claim

What is claimed is:

1 . A method comprising:
receiving a query, at a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor; modifying, at the computer system, the received query to utilize at least one data skipping index to evaluate at least one criterion of the query, wherein the at least one data skipping index comprises an index on at least one attribute of data that excludes at least a portion of those data items that do not meet the at least one criterion, and wherein at least one of the at least one data skipping index or a mapping from the at least one criterion to the at least one data skipping index is generated based on information received from an application programming interface; and evaluating, at the computer system, the query.

2 . The method of claim 1 , wherein the at least one data skipping index is generated by:
receiving from the application programming interface, at the computer system, information defining a data skipping index type; receiving from the application programming interface, at the computer system, information interpreting the at least one criterion; generating, at the computer system, metadata relating to the defined data skipping index type and the defined at least one criterion; and generating, at the computer system, the at least one data skipping index based on the generated metadata.

3 . The method of claim 2 , wherein the received query is represented as an expression tree, and the expression tree is modified by utilizing an optimization rule to label at least one node of the expression tree with a clause representing skipping requirements for the at least one criterion and referring to at least one data skipping index.

4 . The method of claim 3 , wherein the at least one criterion is a Structured Query Language (SQL) predicate.

5 . The method of claim 1 , wherein the at least one data skipping index is generated by:
receiving from the application programming interface, at the computer system, information defining a plurality of data skipping index types; receiving from the application programming interface, at the computer system, information defining a plurality of criteria; generating, at the computer system, metadata relating to each defined data skipping index type and each defined criterion; combining, at the computer system, the metadata relating to each defined data skipping index type and each defined criterion to form metadata relating to the plurality of defined data skipping index types and the plurality of defined criteria; and generating, at the computer system, the at least one data skipping index based on the generated metadata.

6 . The method of claim 5 , wherein the received query is represented as an expression tree, and the expression tree is modified by utilizing a plurality of optimization rules to label each of a plurality of nodes of the expression tree with a clause representing skipping requirements for a criterion and referring to a data skipping index.

7 . The method of claim 6 , wherein each criterion is a Structured Query Language (SQL) predicate.

8 . A system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:
receiving a query; modifying the received query to utilize at least one data skipping index to evaluate at least one criterion of the query, wherein the at least one data skipping index comprises an index on at least one attribute of data that excludes at least a portion of those data items that do not meet the at least one criterion, and wherein at least one of the at least one data skipping index or a mapping from the at least one criterion to the at least one data skipping index is generated based on information received from an application programming interface; and evaluating the query.

9 . The system of claim 8 , wherein the at least one data skipping index is generated by:
receiving from the application programming interface information interpreting a data skipping index type; receiving from the application programming interface information defining the at least one criterion; generating metadata relating to the defined data skipping index type and the defined at least one criterion; and generating the at least one data skipping index based on the generated metadata.

10 . The system of claim 9 , wherein the received query is represented as an expression tree, and the expression tree is modified by utilizing an optimization rule to label at least one node of the expression tree with a clause representing skipping requirements for the at least one criterion and referring to at least one data skipping index.

11 . The system of claim 10 , wherein the at least one criterion is a Structured Query Language (SQL) predicate.

12 . The system of claim 8 , wherein the at least one data skipping index is generated by:
receiving from the application programming interface information defining a plurality of data skipping index types; receiving from the application programming interface information defining a plurality of criteria; generating metadata relating to each defined data skipping index type and each defined criterion; combining the metadata relating to each defined data skipping index type and each defined criterion to form metadata relating to the plurality of defined data skipping index types and the plurality of defined criteria; and generating the at least one data skipping index based on the generated metadata.

13 . The system of claim 12 , wherein the received query is represented as an expression tree, and the expression tree is modified by utilizing a plurality of optimization rules to label each of a plurality of nodes of the expression tree with a clause representing skipping requirements for a criterion and referring to a data skipping index.

14 . The system of claim 13 , wherein each criterion is a Structured Query Language (SQL) predicate.

15 . A computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause a computer to perform a method comprising:
receiving a query; modifying the received query to utilize at least one data skipping index to evaluate at least one criterion of the query, wherein the at least one data skipping index comprises an index on at least one attribute of data that excludes at least a portion of those data items that do not meet the at least one criterion, and wherein at least one of the at least one data skipping index or a mapping from the at least one criterion to the at least one data skipping index is generated based on information received from an application programming interface; and evaluating the query.

16 . The computer program product of claim 15 , wherein the at least one data skipping index is generated by:
receiving from the application programming interface information defining a data skipping index type; receiving from the application programming interface information interpreting the at least one criterion; generating metadata relating to the defined data skipping index type and the defined at least one criterion; and generating the at least one data skipping index based on the generated metadata.

17 . The computer program product of claim 16 , wherein the received query is represented as an expression tree, and the expression tree is mod

Assignees

• IBM

Inventors

• Khazma Guy
• Lushi Gal
• Ta-Shma Paula

Classifications

• G06F16/24534

  Query rewriting; Transformation · CPC title

• G06F16/2282 Primary

  Tablespace storage structures; Management thereof · CPC title

• G06F16/90335

  Query processing · CPC title

• G06F16/24553 Primary

  of query operations · CPC title

• G06F16/9032

  Query formulation · CPC title