Selecting resource configurations for query execution

Patent metadata

Field	Value
Publication number	US-2018060394-A1
Application number	US-201715470843-A
Country	US
Kind code	A1
Filing date	Mar 27, 2017
Priority date	Sep 1, 2016
Publication date	Mar 1, 2018
Grant date	—

Abstract

The configuration of computing resources for executing queries may be selected. A comparison of the configuration of computing resources that executed previous queries may be made to select the configuration of computing resources for a received query. A historical query execution model maybe applied, in some embodiments, to determine a resource configuration for computing resources to execute a query. The computing resources may be selected from available computing resources according to the determined resource configuration.

First claim

What is claimed is:

1 . A system, comprising:
a memory to store program instructions which, if performed by at least one processor, cause the at least one processor to perform a method to at least:
receive a first query directed to one or more data sets;
in response to the receipt of the first query:
evaluate the first query with respect to resource configurations that executed prior queries to determine a resource configuration for executing the query;
select a computing resource to execute the first query from a plurality of differently configured computing resources that execute queries, based, at least in part, on the resource configuration; and
route the first query to the selected computing resource for execution.

2 . The system of claim 1 , wherein the first query indicates an execution limit for the first query, and wherein the selection of the computing resource to execute the first query determines that the computing resource is able to satisfy the execution limit for the first query.

3 . The system of claim 1 , wherein evaluate the first query with respect to resource configurations that executed prior queries comprises:
generate an execution plan for at least one query engine; and apply a historical query execution model to the first query and the at least one query plan to determine the resource configuration for the first query.

4 . The system of claim 3 , wherein the memory and the at least one processor are implemented as part of a managed query service, wherein the one or more data sets are stored as part of a data storage service, wherein the data storage service and the managed query service are implemented as part of a provider network, wherein the first query is one of a plurality of different queries received at the managed query service from different clients of the managed query service, wherein the evaluating, the selecting, and the routing are performed for the different queries, and wherein the method further comprises update the historical query execution model based on respective execution data for the different queries.

5 . A method, comprising:
receiving a first query directed to one or more data sets; in response to receiving the first query:
selecting one or more computing resources to execute the first query based, at least in part, on how the one or more computing resources are configured compared to computing resources used in performing prior queries; and
performing the first query at the selected computing resources with respect to the one or more data sets.

6 . The method of claim 5 , wherein the selecting of the computing resource to execute the first query comprises determining that the computing resource satisfies the execution limit for the first query.

7 . The method of claim 5 , wherein selecting the computing resources to execute the first query comprises:
generating an execution plan for a query engine; and applying a historical query execution model to the first query and the query plan to determine a resource configuration for the computing resources.

8 . The method of claim 7 , further comprising updating the historical query execution model based on execution data generated from the execution of the first query at the selected computing resources.

9 . The method of claim 8 , further comprising updating the historical query execution model based on metadata describing the one or more data sets.

10 . The method of claim 7 , wherein determining the resource configuration for the computing resources comprises determining configuration settings for a query engine to execute the first query.

11 . The method of claim 7 , wherein determining the resource configuration for the computing resources comprises selecting one query engine from a plurality of query engines to execute the first query.

12 . The method of claim 5 , wherein selecting the one or more computing resources to execute the first query comprises including capacity to execute one or more subsequent queries at the selected computing resources based, at least in part, on a concurrent query pattern for a submitter of the first query.

13 . The method of claim 5 , further comprising:
prior to receiving the first query:
evaluating a resource provisioning model generated from prior executed queries to determine at least one computing resource to provision, including the one or more computing resources; and
provisioning the determined at least one computing resource for executing queries.

14 . A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:
receiving a first query directed to one or more data sets; in response to the receipt of the first query:
evaluating the first query with respect to resource configurations that executed prior queries to determine a resource configuration for executing the first query;
selecting a computing resource to execute the first query from a plurality of differently configured computing resources that execute queries, based, at least in part, on the resource configuration; and
performing the first query at the selected computing resource.

15 . The non-transitory, computer-readable storage medium of claim 14 , wherein the first query indicates an execution limit for the first query, and wherein the selection of the computing resource to execute the first query determines that the computing resource is able to satisfy the execution limit for the first query.

16 . The non-transitory, computer-readable storage medium of claim 14 , wherein, in evaluating the first query with respect to resource configurations that executed prior queries, the program instructions cause the one or more computing devices to implement:
generating an execution plan for query engine; and applying a historical query execution model to the first query and the query plan to determine the resource configuration for the first query.

17 . The non-transitory, computer-readable storage medium of claim 16 , wherein determining the resource configuration for the computing resources comprises determining a number of slots included in the computing resources.

18 . The non-transitory, computer-readable storage medium of claim 14 , wherein the program instructions cause the one or more computing devices to implement:
prior to receiving the first query:
detect a provisioning event to provision a pool of computing resources that includes the one or more computing resources;
evaluating a resource provisioning model generated from prior executed queries to determine resource plurality of computing resources to provision for the pool, including the one or more computing resources; and
provisioning the determined computing resources for the pool.

19 . The non-transitory, computer-readable storage medium of claim 18 , wherein the program instructions cause the one or more computing devices to further implement updating the resource provisioning model based on execution data generated from the execution of the first query at the selected computing resources.

20 . The non-transitory, computer-readable storage medium of claim 14 , wherein evaluating the resource provisioning model generated from prior executed queries to determine the at least one computing resource to provision comprises performing a time-based evaluation of the prior executed queries to determine the at least one computing resource for a time period associat

Assignees

• Amazon Tech Inc

Inventors

• Gawande Pratik Bhagwat
• Maru Sumeetkumar Veniklal
• Kalathuru Bhargava Ram
• Fang Jian
• Wu Xing
• Yue Yuanyuan
• Hocanin Turkay Mert
• Denton Jason Douglas
• Natali Luca
• Pathak Rahul Sharma
• Sinha Abhishek Rajnikant
• Tangamyan Armen
• Jiang Yufeng
• Beitchman Marc Howard
• Caldwell Andrew Edward

Classifications

• H04L67/1029

  using data related to the state of servers by a load balancer · CPC title

• G06F16/24549

  Run-time optimisation · CPC title

• G06F2209/503

  Resource availability · CPC title

• H04L67/1031

  Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests · CPC title

• G06F16/20

  of structured data, e.g. relational data · CPC title