Resource provisioning in database systems

Patent metadata

Field	Value
Publication number	US-11914602-B2
Application number	US-202218050255-A
Country	US
Kind code	B2
Filing date	Oct 27, 2022
Priority date	Apr 10, 2019
Publication date	Feb 27, 2024
Grant date	Feb 27, 2024

Abstract

Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

First claim

What is claimed is:

1. A method comprising:
determining, by one or more processors, whether one or more statements of a query plan are to be performed on a row of a table; and
in response to determining whether the one or more statements are to be performed on the row of the table, assigning the one or more statements respectively to one or more nodes of a plurality of nodes of an execution platform.

2. The method of claim 1 , further comprising:
determining a set of actions to be executed on the table in response to a trigger event;
determining the query plan for executing the set of actions, the query plan comprising a plurality of statements to be executed;
determining whether execution of the set of actions is complete; and
storing a record of each completed action of the set of actions.

3. The method of claim 2 , wherein the trigger event comprises at least one of an insert, update, or delete operation.

4. The method of claim 1 , further comprising retrieving a set of actions from a queue, wherein the queue is stored across one or more of a plurality of shared storage devices that are separate from the execution platform.

5. The method of claim 4 , further comprising scaling a number of available execution nodes in the execution platform up or down based on a number of sets of actions waiting in the queue.

6. The method of claim 1 , further comprising deserializing the plurality of statements to resolve a context for the plurality of statements, wherein the context comprises at least one of an account identification, a role, or a user for executing a task.

7. The method of claim 1 , comprising generating a Structured Query Language (SQL) job, wherein the one or more statements must be processed to complete the SQL job.

8. The method of claim 7 , further comprising dividing the processing of the one or more statements based on metadata, wherein the metadata pertains to database data that is stored across a plurality of shared storage devices, and wherein the metadata is stored separately from the database data.

9. The method of claim 1 , further comprising:
predicting a time required to execute a given statement of the one or more statements;
determining a current processing delay associated with the one or more nodes based on the predicted time; and
in response to determining that the current processing delay exceeds a threshold value, creating a new node for processing the given statement.

10. The method of claim 1 , wherein assigning the one or more statements to one or more nodes further comprises:
assigning a first statement to a first node, the assigning the first statement to the first node comprises determining that the first node includes cached data associated with the first statement; and
assigning a second statement to a second node, the second node being distinct from the first node.

11. The method of claim 1 , further comprising prioritizing user queries over internal jobs of the execution platform.

12. The method of claim 1 , further comprising:
determining a future workload of the plurality of nodes; and
scheduling each of the one or more statements based on the future workload of the plurality of nodes.

13. A system comprising:
a plurality of storage devices collectively storing database data; and
a service manager comprising a software program stored in memory and executed by one or more processors to perform operations comprising:
determining whether one or more statements of a query plan are to be performed on a row of a table; and
in response to determining whether the one or more statements are to be performed on the row of the table, assigning the one or more statements respectively to one or more nodes of a plurality of nodes of an execution platform.

14. The system of claim 13 , wherein the operations further comprise:
determining a set of actions to be executed on the table in response to a trigger event;
determining the query plan for executing the set of actions, the query plan comprising a plurality of statements to be executed;
determining whether execution of the set of actions is complete; and
storing a record of each completed action of the set of actions.

15. The system of claim 13 , wherein the operations further comprise:
determining whether the one or more nodes of the plurality of nodes of the execution platform has finished executing the query plan.

16. The system of claim 13 , wherein the operations further comprise scaling a number of available execution nodes in the execution platform up or down based on a number of sets of tasks within a queue.

17. The system of claim 13 , wherein the operations further comprise:
deserializing the one or more statements to resolve a context for a set of actions, wherein the context comprises at least one of an account identification, a role, or a user for executing the set of actions; and
dividing processing of the one or more statements based on metadata, wherein the metadata pertains to the database data and is stored separately from the database data.

18. The system of claim 13 , wherein the operations further comprise retrieving a set of actions from a queue, wherein the queue is stored across one or more of a plurality of shared storage devices that are separate from the execution platform.

19. The system of claim 13 , the operations comprising determining a set of actions to be executed on the database table in response to a trigger event, wherein the trigger event comprises at least one of an insert, update, or delete operation.

20. The system of claim 13 , wherein the operations comprise generating a Structured Query Language (SQL) job, wherein the one or more statements must be processed to complete the SQL job.

21. The system of claim 13 , wherein the operations further comprise:
predicting a time required to execute a given statement of the one or more statements;
determining a current processing delay associated with the one or more nodes based on the predicted time; and
in response to determining that the current processing delay exceeds a threshold value, creating a new node for processing the given statement.

22. A non-transitory computer-readable medium comprising instructions, which when executed by at least one processor, cause the at least one processor to perform operations comprising:
determining whether one or more statements of a query plan are to be performed on a row of a table; and
in response to determining whether the one or more statements are to be performed on the row of the table, assigning the one or more statements respectively to one or more nodes of a plurality of nodes of an execution platform.

23. The non-transitory computer-readable medium of claim 22 , the operations further comprises:
determining a set of actions to be executed on the table in response to a trigger event;
determining the query plan for executing the set of actions, the query plan comprising a plurality of statements to be executed;
determining whether execution of the set of actions is complete; and
storing a record of each completed action of the set of actions.

24. The non-transitory computer-readable medium of claim 22 , the operations further comprising:
retrieving a set of actions from a queue, wherein the queue stores the set of actions across one or more of a plurality of shared storage devices that are separate from the execution platform.

25. The non-transitory compute

Assignees

• Snowflake Inc

Inventors

• Rosen Jeffrey
• Munir Abdul
• Yan Jiaqi
• Waddington William
• Rajaperumal Prasanna
• Cruanes Thierry

Classifications

• G06F16/2465 Primary

  Query processing support for facilitating data mining operations in structured databases · CPC title

• G06F9/505

  considering the load · CPC title

• G06F9/5077

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F16/24535

  of sub-queries or views · CPC title

• G06F16/24542

  Plan optimisation · CPC title