Techniques for improving resource utilization in a microservices architecture via priority queues

Patent metadata

Field	Value
Publication number	US-11743200-B2
Application number	US-202117409139-A
Country	US
Kind code	B2
Filing date	Aug 23, 2021
Priority date	Aug 23, 2021
Publication date	Aug 29, 2023
Grant date	Aug 29, 2023

Abstract

In various embodiments, a flexible queue application allocates messages stored in priority queues to clients. In operation, the flexible queue application receives, from a client, a request to allocate a message from a priority queue. At least a first message and a second message are stored in the priority queue, and the priority of the first message is higher than the priority of the second message. The flexible queue application determines that the first message is pending but does not satisfy an allocation constraint. The flexible queue allocation then determines that the second message is pending and satisfies the allocation constraint. The flexible queue application allocates the second message to the client. Advantageously, because the flexible queue application can adapt the priority-based ordering of priority queues based on allocation constraints, the flexible queue application can efficiently enforce resource-related constraints when allocating messages from priority queues.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, from a first client, a first request to allocate a message from a priority queue;
determining that a first message that is stored in the priority queue and is pending does not satisfy a first allocation constraint, wherein a first priority is associated with the first message;
determining that a second message that is stored in the priority queue and is pending satisfies the first allocation constraint, wherein the first priority associated with the first message is higher than a second priority associated with the second message; and
based on the second message satisfying the first allocation constraint, overriding the first priority associated with the first message, and allocating the second message to the first client.

2. The computer-implemented method of claim 1 , wherein determining that the first message does not satisfy the first allocation constraint comprises comparing a first exclusivity key value associated with the first message to a second exclusivity key value associated with a third message that is stored in the priority queue and is running.

3. The computer-implemented method of claim 2 , further comprising:
deleting the third message from the priority queue;
receiving, from the first client, a second request to allocate a message from the priority queue; and
allocating the first message to the first client.

4. The computer-implemented method of claim 1 , wherein determining that the first message does not satisfy the first allocation constraint comprises comparing at least one key-value pair specified via the first request to zero or more key-value pairs associated with the first message.

5. The computer-implemented method of claim 1 , wherein the first message is associated with a first exclusivity key value, and wherein determining that the first message does not satisfy the first allocation constraint comprises determining that a third message that is stored in the priority queue and is running is associated with the first exclusivity key value or determining that the first message is not associated with a first key-value pair specified via the first request.

6. The computer-implemented method of claim 1 , wherein the first allocation constraint is associated with a processor type, a function identifier, or a project identifier.

7. The computer-implemented method of claim 1 , further comprising:
receiving, from a second client, a second request to allocate a message from the priority queue;
determining that the first message is pending and satisfies a second allocation constraint; and
allocating the first message to the second client.

8. The computer-implemented method of claim 1 , further comprising:
receiving, from the first client, a second request to postpone the second message for a first duration of time;
updating a state associated with the second message to indicate that the second message is invisible; and
after determining that the first duration of time has elapsed, updating the state associated with the second message to indicate that the second message is pending.

9. The computer-implemented method of claim 1 , wherein the second message specifies a task and allocating the second message to the first client causes the first client to execute the task.

10. The computer-implemented method of claim 1 , wherein the first client comprises a microservice, a software application, or a function.

11. One or more non-transitory computer readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
receiving, from a first client, a first request to allocate a message from a priority queue;
determining that a first message that is stored in the priority queue and is pending does not satisfy a first allocation constraint, wherein a first priority is associated with the first message;
determining that a second message that is stored in the priority queue and is pending satisfies the first allocation constraint, wherein the first priority associated with the first message is higher than a second priority associated with the second message; and
based on the second message satisfying the first allocation constraint, overriding the first priority associated with the first message, and allocating the second message to the first client.

12. The one or more non-transitory computer readable media of claim 11 , wherein determining that the first message does not satisfy the first allocation constraint comprises comparing a first exclusivity key value associated with the first message to a second exclusivity key value associated with a third message that is stored in the priority queue and is running.

13. The one or more non-transitory computer readable media of claim 12 , further comprising:
deleting the third message from the priority queue;
receiving, from the first client, a second request to allocate a message from the priority queue; and
allocating the first message to the first client.

14. The one or more non-transitory computer readable media of claim 11 , wherein determining that the first message does not satisfy the first allocation constraint comprises comparing at least one key-value pair specified via the first request to zero or more key-value pairs associated with the first message.

15. The one or more non-transitory computer readable media of claim 14 , wherein the at least one key-value pair specifies at least one of a processor type, a function identifier, or a project identifier.

16. The one or more non-transitory computer readable media of claim 11 , wherein the first message is associated with a first exclusivity key value, and wherein determining that the first message does not satisfy the first allocation constraint comprises determining that a third message that is stored in the priority queue and is running is associated with the first exclusivity key value or determining that the first message is not associated with a first key-value pair specified via the first request.

17. The one or more non-transitory computer readable media of claim 11 , further comprising:
receiving, from a second client, a second request to allocate a message from the priority queue;
determining that the first message is pending and satisfies a second allocation constraint; and
allocating the first message to the second client.

18. The one or more non-transitory computer readable media of claim 11 , further comprising:
determining that the first client has not successfully processed the second message within a lease duration of time;
updating a state associated with the second message to indicate that the second message is invisible; and
after an invisibility duration of time, updating the state associated with the second message to indicate that the second message is pending.

19. The one or more non-transitory computer readable media of claim 11 , wherein the second message specifies a task and allocating the second message to the first client causes the first client to execute the task.

20. A system, comprising:
one or more memories storing instructions; and
one or more processors coupled to the one or more memories that, when executing the instructions, perform the steps of:
receiving, from a first client, a first request to allocate a message from a priority queue;
determining that a first message that is stored in the priority queue and is pending does not satisfy a first allocation constraint, wherein a first

Assignees

• Netflix Inc

Inventors

• Christidis Konstantinos
• Ramkumar Aravindan
• Reddy Poorna Chandra Tejashvi
• Zhao Chunyun

Classifications

• H04L47/522 Primary

  Dynamic queue service slot or variable bandwidth allocation · CPC title

• H04L47/2466

  using signalling traffic · CPC title

• H04L47/2475

  for supporting traffic characterised by the type of applications · CPC title

• G06F9/546 Primary

  Message passing systems or structures, e.g. queues · CPC title