Dynamic query allocation to virtual warehouses

Patent metadata

Field	Value
Publication number	US-12386840-B2
Application number	US-202318140260-A
Country	US
Kind code	B2
Filing date	Apr 27, 2023
Priority date	Jul 13, 2021
Publication date	Aug 12, 2025
Grant date	Aug 12, 2025

Abstract

Methods, systems, and apparatuses for managing and selecting virtual warehouses for execution of queries on one or more data warehouses are described herein. A request to execute a query may be received. An execution plan, for the query, may be identified. A processing complexity for the query may be predicted based on the query and the execution plan. A plurality of virtual warehouses may be identified. An operating status and processing capabilities of the plurality of virtual warehouses may be determined. A subset of the plurality of virtual warehouses may be selected based on the processing complexity, the operating status of the plurality of virtual warehouses, and the processing capabilities of the plurality of virtual warehouses. The query may be executed on one of the subset of the plurality of virtual warehouses.

First claim

What is claimed is:

1. A computing device comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the computing device to:
receive, from a user device, a request to execute a query on at least one of a plurality of data warehouses;
modify the query by adding, to a metadata field of the query, information that identifies a user of the user device;
identify a plurality of virtual warehouses, wherein each of the plurality of virtual warehouses comprises a respective set of computing resources configured to:
execute one or more queries with respect to at least a portion of the plurality of data warehouses;
collect results from the one or more queries; and
provide, to the user device, access to the collected results;

select a first virtual warehouse of the plurality of virtual warehouses based on one or more of:
an operating status of the plurality of virtual warehouses,
processing capabilities of the plurality of virtual warehouses, or
a predicted processing complexity of the query;

cause the first virtual warehouse to execute the query;
receive, based on execution of the query by the first virtual warehouse:
query results that comprise the metadata field; and
costs data;

extract the information that identifies the user of the user device from the metadata field in the query results; and
output, based on the costs data and the extracted information in the metadata field of the query results, query cost information corresponding to the information.

2. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:
identify an execution plan for the query by determining one or more sub-queries to be executed with respect to one or more of the plurality of data warehouses; and
determine, based on the query and the execution plan, the predicted processing complexity of the query.

3. The computing device of claim 1 , wherein the costs data corresponds to a quantity of data processed by the first virtual warehouse.

4. The computing device of claim 1 , wherein the costs data corresponds to a time period during which the first virtual warehouse executed the query.

5. The computing device of claim 1 , wherein the output query cost information comprises a report of query costs, associated with the information, over a period of time.

6. The computing device of claim 1 , wherein the output query cost information comprises an indication of aggregated query cost information for a plurality of different organizations.

7. The computing device of claim 1 , wherein the query cost information is based on a size of the first virtual warehouse.

8. A method comprising:
receiving, from a user device, a request to execute a query on at least one of a plurality of data warehouses;
modifying the query by adding, to a metadata field of the query, information that identifies a user of the user device;
identifying a plurality of virtual warehouses, wherein each of the plurality of virtual warehouses comprises a respective set of computing resources configured to:
execute one or more queries with respect to at least a portion of the plurality of data warehouses;
collect results from the one or more queries; and
provide, to the user device, access to the collected results;

selecting a first virtual warehouse of the plurality of virtual warehouses based on one or more of:
an operating status of the plurality of virtual warehouses,
processing capabilities of the plurality of virtual warehouses, or
a predicted processing complexity of the query;

causing the first virtual warehouse to execute the query;
receiving, based on execution of the query by the first virtual warehouse:
query results that comprise the metadata field; and
costs data;

extracting the information that identifies the user of the user device from the metadata field in the query results; and
outputting, based on the costs data and the extracted information in the metadata field of the query results, query cost information corresponding to the information.

9. The method of claim 8 , further comprising:
identifying an execution plan for the query by determining one or more sub-queries to be executed with respect to one or more of the plurality of data warehouses; and
determine, based on the query and the execution plan, the predicted processing complexity of the query.

10. The method of claim 8 , wherein the costs data corresponds to a quantity of data processed by the first virtual warehouse.

11. The method of claim 8 , wherein the costs data corresponds to a time period during which the first virtual warehouse executed the query.

12. The method of claim 8 , wherein output query cost information comprises a report of query costs, associated with the information, over a period of time.

13. The method of claim 8 , wherein the output query cost information comprises an indication of aggregated query cost information for a plurality of different organizations.

14. The method of claim 8 , wherein the query cost information is based on a size of the first virtual warehouse.

15. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause a computing device to:
receive, from a user device, a request to execute a query on at least one of a plurality of data warehouses;
modify the query by adding, to a metadata field of the query, information that identifies a user of the user device;
identify a plurality of virtual warehouses, wherein each of the plurality of virtual warehouses comprises a respective set of computing resources configured to:
execute one or more queries with respect to at least a portion of the plurality of data warehouses;
collect results from the one or more queries; and
provide, to the user device, access to the collected results;
select a first virtual warehouse of the plurality of virtual warehouses based on one or more of:
an operating status of the plurality of virtual warehouses,
processing capabilities of the plurality of virtual warehouses, or
a predicted processing complexity of the query;

cause the first virtual warehouse to execute the query;
receive, based on execution of the query by the first virtual warehouse:
query results that comprise the metadata field; and
costs data;

extract the information that identifies the user of the user device from the metadata field in the query results; and
output, based on the costs data and the extracted information in the metadata field of the query results, query cost information corresponding to the information.

16. The computer-readable media of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:
identify an execution plan for the query by determining one or more sub-queries to be executed with respect to one or more of the plurality of data warehouses; and
determine, based on the query and the execution plan, the predicted processing complexity of the query.

17. The computer-readable media of claim 15 , wherein the costs data corresponds to a quantity of data processed by the first virtual warehouse.

18. The computer-readable media of claim 15 , wherein the costs data corresponds to a time period during which the first virtual warehouse executed the query.

19. The computer-readable media of claim 15 , wherein the output query cost information comprises a report of query cos

Assignees

• Capital One Services Llc

Inventors

• Kandukuri Praveen
• Salim Syed Shamaz
• Hardatt Karamchandradatt
• Gurram Nagender
• Bharathan Ganesh
• Batra Yudhish

Classifications

• G06F16/256

  in federated or virtual databases · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F9/5005

  to service a request · CPC title

• G06F16/24575 Primary

  using context · CPC title

• G06F16/283 Primary

  Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP · CPC title