Sharing compiled code for executing queries across query engines

Patent metadata

Field	Value
Publication number	US-11853301-B1
Application number	US-201916370614-A
Country	US
Kind code	B1
Filing date	Mar 29, 2019
Priority date	Mar 29, 2019
Publication date	Dec 26, 2023
Grant date	Dec 26, 2023

Abstract

Compiled portions of code generated to perform a query plan at a query engine may be shared with other query engines. A data store, separate from the query engines, may store compiled portions of query code generated for different queries. If a query engine does not have a locally stored compiled portion of query code, then the separate data store may be accessed in order to obtain a compiled portion of query code, allowing reuse of compiled query code across different queries engines for queries directed to different databases.

First claim

What is claimed is:

1. A system, comprising:
at least one processor; and
a memory, storing program that when executed by the at least one processor cause the at least one processor to implement a query engine;
the query engine, configured to:
receive a query to a database;
generate a plurality of portions of code for a query execution plan to perform the query;
before compiling the plurality of portions of code:
determine that a portion of code of the plurality of portions of code for the query execution plan is not compiled and that the portion of code is not stored in a local store of one or more portions of compiled code at the query engine,
obtain a compiled version of the portion of code of the plurality of portions of code from a global store, wherein the compiled version of the portion of code of the plurality of portions of code stored in the global store was previously generated for a prior query execution plan to perform a prior query received at a second query engine, and
store the compiled version of the portion of code obtained from the global store in the local store; and

use the compiled version of the portion of code of the plurality of portions of code obtained from the global store and stored in the local store instead of compiling the portion of code when executing the query execution plan to perform the query.

2. The system of claim 1 , wherein the query engine is further configured to:
receive a second query to the database;
generate another plurality of portions of code to execute another plan to perform the second query;
before compiling the other plurality of portions of code:
determine that another portion of code of the other plurality of portions of code to execute the plan is not compiled and that the other portion of code is not stored in the local store, and
evaluate the global store of one or more compiled portions of code at the query engine to determine that the other portion of code of the other plurality of portions of code to execute the plan is not compiled and stored in the global store;

compile the other portion of code of the other plurality of portions of code; and
use the compiled version of the other portion of code when executing the other plan to perform the second query.

3. The system of claim 1 , wherein the query engine is implemented as part of processing cluster for a data warehouse service of a provider network, wherein the database is stored as part of the data warehouse service, and wherein the different query is directed to a different database hosted in the data warehouse service.

4. The system of claim 1 , wherein the query is different than the prior query received at the second query engine.

5. A method, comprising:
generating, by a first query engine, a plurality of portions of code for a query execution plan to perform a query received at the first query engine;
determining, by the first query engine, that a portion of code of the plurality of portions of code to execute the plan is not compiled and that the portion of code is not stored in a first data store of one or more portions of compiled code at the first query engine;
obtaining, by the first query engine, a compiled version of the portion of code of the plurality of portions of code from a second data store, wherein the compiled version of the portion of code of the plurality of portions of code stored in the second data store was previously generated for a prior query execution plan to perform a prior query received at a second query engine,
storing, by the first query engine, the compiled version of the portion of code obtained from the second data store in the first data store; and
executing, by the first query engine, the compiled version of the portion of code of the plurality of portions of code obtained from the second data store and stored in the first data store when executing the query execution plan to perform the query.

6. The method of claim 5 , wherein evaluating the first data store of the one or more portions of compiled code at the first query engine comprises:
generating a unique identifier for the portion the code of the plurality of portions of code;
using the unique identifier as a lookup key at the first data store to determine that the portion of code to execute the plan is not compiled and stored in the first data store; and
wherein obtaining the compiled version of the portion of code of the plurality of portions of code from the second data store comprises using the unique identifier as a lookup key at the second data store to obtain the compiled version of the portion of code of the plurality of portions of code.

7. The method of claim 5 , further comprising:
generating, by the first query engine, another portions of code to execute another plan to perform a second query received at the first query engine;
determining, by the first query engine, that the other portion of code of the other plurality of portions of code to execute the plan is not compiled and that the other portion of code is not stored in the first data store;
determining, by the first query engine, that the other portion of code of the other plurality of portions of code to execute the plan is not compiled and stored in the second data store;
compiling, by the first query engine, the other portion of code of the other plurality of portions of code; and
executing, by the first query engine, the compiled version of the one other portion of code when executing the other plan to perform the second query.

8. The method of claim 7 , further comprising storing, by the first query engine, the compiled version of the other portion of code of the other plurality of portions of code to the second data store.

9. The method of claim 7 , further comprising storing the compiled version of the other portion of code of the other plurality of portions of code to the second data store according to a determined size of the compiled version of the other portion of code of the other plurality of portions of code.

10. The method of claim 7 , further comprising:
storing, by the first query engine, the other portion of code of the other plurality of portions of code to a third data store;
accessing, by a query execution code compilation service, the third data store to obtain the other portion of code of the other plurality of portions of code;
compiling, by the query execution code compilation service, the other portion of code of the other plurality of portions of code; and
storing, by the query execution code compilation service, to the second data store.

11. The method of claim 10 , wherein a compilation setting applied when compiling the other portion of code of the other plurality of portions of code by the query execution code compilation service is different than the same compilation setting applied when compiling the other portion of code of the other plurality of portions of code by the first query engine.

12. The method of claim 5 , further comprising:
before receiving the query:
detecting a warming event for the second data store;
identifying the other portion of code of the other plurality of portions of code as a code segment compiled and stored in a third data store;
compiling the identified other portion of code of the other plurality of portions of code; and
storing the compiled version of the identified other portion of code of the other plurality of portions of code in the second data store.

13. The method of claim 12 , wherein the warming event is triggered by a deployment of a different version of an application implementing the first query engine at a host for the first query engine.

Assignees

• Amazon Tech Inc

Inventors

• Pandis Ippokratis
• Chainani Naresh
• Chinta Kiran Kumar
• Govindaraju Venkatraman
• Caldwell Andrew Edward
• Muralimanohar Naveen
• Grund Martin
• Nagel Fabian Oliver
• Armenatzoglou Nikolaos

Classifications

• G06F16/24552 Primary

  Database cache management · CPC title

• G06F16/2255

  Hash tables · CPC title

• G06F16/2443

  Stored procedures · CPC title

• G06F16/2445

  Data retrieval commands; View definitions · CPC title

• G06F16/2458

  Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries · CPC title