Asynchronous task queue configuration in a database system

Patent metadata

Field	Value
Publication number	US-12530229-B2
Application number	US-202418670459-A
Country	US
Kind code	B2
Filing date	May 21, 2024
Priority date	Jul 28, 2023
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

Techniques for configuring an asynchronous task queue are disclosed. An example method includes enqueuing, by at least one hardware processor of a first computing node, a task in a task queue. The at least one hardware processor detects a lease request for the task. The lease request is received from a second computing node. The at least one hardware processor initiates a lease of the task to the second computing node. The at least one hardware processor updates a state of the task based on the lease. The at least one hardware processor determines whether to dequeue the task based on a task processing update received from the second computing node.

First claim

What is claimed is:

1 . A method comprising:
enqueuing, by at least one hardware processor of a first computing node, a task in a task queue; detecting a lease request for the task, the lease request received from a second computing node; initiating a lease of the task to the second computing node; updating a state of the task based on the lease; determining whether to dequeue the task based on a task processing update received from the second computing node; and responsive to a dequeue request received from the second computing node and indicating completion of the task, dequeuing the task from the task queue.

2 . The method of claim 1 , comprising:
updating the state of the task to a lease state, the lease state indicating the task is leased to the second computing node for processing.

3 . The method of claim 2 , comprising:
initiating a lease timer for a pre-configured period, the lease timer starting at a time the lease is initiated.

4 . The method of claim 3 , wherein the first computing node receives the dequeue request prior to expiration of the lease timer.

5 . The method of claim 1 , comprising:
sharding the task queue into a plurality of shards; and storing the task in a first shard of the plurality of shards.

6 . The method of claim 5 , comprising:
decoding an enqueue request received from a data process, the enqueue request including a second task; and enqueuing the second task in a second shard of the plurality of shards.

7 . The method of claim 6 , comprising:
enqueuing one or more additional tasks received after the second task, wherein the one or more additional tasks are enqueued into corresponding shards of the plurality of shards in a round-robin configuration.

8 . The method of claim 1 , comprising:
enqueuing a second lease request received from a third computing node for a second task in the task queue; and responsive to the enqueuing of the second lease request; initiating a second lease of the second task to the third computing node; initiating a lease timer for a pre-configured period, the lease timer starting a time the second lease is initiated; and changing a state of the second task from a ready state to a leased state.

9 . The method of claim 8 , comprising:
detecting the lease timer has expired; performing a revocation of the second lease; and changing the state of the second task from the leased state to the ready state.

10 . A system comprising:
at least one hardware processor of a first computing node; and at least one memory storing instructions that cause the at least one hardware processor to perform operations comprising:
enqueuing a task in a task queue;
detecting a lease request for the task, the lease request received from a second computing node;
initiating a lease of the task to the second computing node;
updating a state of the task based on the lease;
determining whether to dequeue the task based on a task processing update received from the second computing node; and
responsive to a dequeue request received from the second computing node and indicating completion of the task, dequeuing the task from the task queue.

11 . The system of claim 10 , the operations comprising:
updating the state of the task to a lease state, the lease state indicating the task is leased to the second computing node for processing.

12 . The system of claim 11 , the operations comprising:
initiating a lease timer for a pre-configured period, the lease timer starting at a time the lease is initiated.

13 . The system of claim 12 , wherein the first computing node receives the dequeue request prior to expiration of the lease timer.

14 . The system of claim 10 , the operations comprising:
sharding the task queue into a plurality of shards; and storing the task in a first shard of the plurality of shards.

15 . The system of claim 14 , the operations comprising:
decoding an enqueue request received from a data process, the enqueue request including a second task; and enqueuing the second task in a second shard of the plurality of shards.

16 . The system of claim 15 , the operations comprising:
enqueuing one or more additional tasks received after the second task, wherein the one or more additional tasks are enqueued into corresponding shards of the plurality of shards in a round-robin configuration.

17 . The system of claim 10 , the operations comprising:
enqueuing a second lease request received from a third computing node for a second task in the task queue; and responsive to the enqueuing of the second lease request:
initiating a second lease of the second task to the third computing node;
initiating a lease timer for a pre-configured period, the lease timer starting a time the second lease is initiated; and
changing a state of the second task from a ready state to a leased state.

18 . The system of claim 17 , the operations comprising:
detecting the lease timer has expired; performing a revocation of the second lease; and changing the state of the second task from the leased state to the ready state.

19 . A computer-storage medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:
enqueuing a task in a task queue of a first computing node; detecting a lease request for the task, the lease request received from a second computing node; initiating a lease of the task to the second computing node; updating a state of the task based on the lease; determining whether to dequeue the task based on a task processing update received from the second computing node; and responsive to a dequeue request received from the second computing node and indicating completion of the task, dequeuing the task from the task queue.

20 . The computer-storage medium of claim 19 , the operations comprising:
updating the state of the task to a lease state, the lease state indicating the task is leased to the second computing node for processing.

21 . The computer-storage medium of claim 20 , the operations comprising:
initiating a lease timer for a pre-configured period, the lease timer starting at a time the lease is initiated.

22 . The computer-storage medium of claim 21 , wherein the first computing node receives the dequeue request prior to expiration of the lease timer.

23 . The computer-storage medium of claim 19 , the operations comprising:
sharding the task queue into a plurality of shards; and storing the task in a first shard of the plurality of shards.

24 . The computer-storage medium of claim 23 , the operations comprising:
decoding an enqueue request received from a data process, the enqueue request including a second task; and enqueuing the second task in a second shard of the plurality of shards.

25 . The computer-storage medium of claim 24 , the operations comprising:
enqueuing one or more additional tasks received after the second task, wherein the one or more additional tasks are enqueued into corresponding shards of the plurality of shards in a round-robin configuration.

26 . The computer-storage medium of claim 19 , the operations comprising:
enqueuing a second lease request received from a third computing node for a second task in the task queue; and responsive to the enqueuing of the second lease request:
initiating a second lease of

Assignees

• Snowflake Inc

Inventors

• Kliot Gabriel
• Xie Ruji
• Muralidhar Subramanian
• Waddington William

Classifications

• G06F9/5038

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06F2209/483

  Multiproc · CPC title

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title