Methods and systems for processing requests using load-dependent throttling

Patent metadata

Field	Value
Publication number	US-12175288-B2
Application number	US-202318486254-A
Country	US
Kind code	B2
Filing date	Oct 13, 2023
Priority date	Feb 18, 2022
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

Methods and systems for processing requests with load-dependent throttling. The system compares a count of active job requests being currently processed for a user associated with a new job request with an active job cap number for that user. When the count of active job requests being currently processed for that user does not exceed the active job cap number specific to that user, the job request is added to an active job queue for processing. However, when the count of active job requests being currently processed for that user exceeds the active job cap number, the job request is placed on a throttled queue to await later processing when an updated count of active job requests being currently processed for that user is below the active job cap number. Once the count is below the cap, the throttle request is moved to the active job queue for processing.

First claim

The invention claimed is:

1. A computing device, comprising:
one or more processors;
one or more data storage units; and
an application stored in the one or more data storage units and having processor-readable instructions that, when executed by the one or more processors, are to cause the one or more processors to:
receive a job request to implement one or more actions, the job request associated with a particular user;
compare a count of active job requests being currently processed for the particular user with an active job cap number for that user; and
selectively process the job request based on the comparison of the count of active job requests currently processed for the particular user with the active job cap number for that user,
wherein when the count of active job requests being currently processed for the particular one of the users does not exceed the active job cap number, the job request is added to an active job queue for processing, and
wherein when the count of active job requests being currently processed for the particular user exceeds the active job cap number, the job request is placed on a throttled queue to await later processing when an updated count of active job requests being currently processed for the particular user is below the active job cap number for that user.

2. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, are to cause the one or more processors to later process the job request added to the throttled queue by transferring the job request from the throttled queue to the active job queue.

3. The computing device of claim 2 , wherein the instructions, when executed by the one or more processors, are to further cause the processors to determine the updated count of active job requests being currently processed for the particular user in response to detecting an unthrottling trigger event.

4. The computing device of claim 1 , wherein the active job cap number for the particular user is associated with a feature set corresponding to the job request.

5. The computing device of claim 1 , wherein the active job queue includes, for each job request in the active job queue, a user identifier identifying a user associated with that job request.

6. The computing device of claim 5 , wherein the instructions, when executed by the one or more processors, are to further cause the one or more processors to determine the count of active job requests being currently processed for the particular user by counting a number of user identifiers in the active job queue identifying that user.

7. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, are to cause the one or more processors to place the job request on the throttled queue on condition that the one or more processing units determine that the job request is not a high priority job request.

8. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, are to further cause the one or more processors to:
receive a second job request to implement a second action associated with the particular user;
determine that a current count of active job requests being processed for the particular user exceeds the active job cap number for that user;
determine that the second job request is a high priority job request; and
in response, add the second job request to the active job queue instead of the throttled queue.

9. The computing device of claim 1 , wherein the job request is generated responsive to detecting a trigger condition associated with the action.

10. A computer-implemented method comprising:
receiving a job request to implement one or more actions, the job request associated with a particular user;
comparing a count of active job requests being currently processed for the particular user with an active job cap number for that user; and
selectively processing the job request based on the comparison of the count of active job requests currently processed for the particular user with the active job cap number for that user,
wherein when the count of active job requests being currently processed for the particular user does not exceed the active job cap number, the job request is added to an active job queue for processing, and
wherein when the count of active job requests being currently processed for the particular user exceeds the active job cap number, the job request is placed on a throttled queue to await later processing when an updated count of active job requests being currently processed for the particular user is below the active job cap number for that user.

11. The method of claim 10 , wherein the later processing of the job request added to the throttled queue includes transferring the job request from the throttled queue to the active job queue.

12. The method of claim 11 , wherein determining the updated count of active job requests being currently processed for the particular user occurs in response to detecting an unthrottling trigger event.

13. The method of claim 10 , wherein the active job cap number for the particular user is associated with a feature set corresponding to the job request.

14. The method of claim 10 , wherein the active job queue includes, for each job request in the active job queue, a user identifier identifying a user associated with that job request.

15. The method of claim 14 , further comprising determining the count of active job requests being currently processed for the particular user by counting a number of user identifiers in the active job queue identifying that user.

16. The method of claim 10 , wherein placing the job request on the throttled queue requires, as a precondition, determining that the job request is not a high priority job request.

17. The method of claim 10 , further comprising:
receiving a second job request to implement a second action associated with the particular user;
determining that a current count of active job requests being processed for the particular user exceeds the active job cap number for that user;
determining that the second job request is a high priority job request; and
in response, adding the second job request to the active job queue instead of the throttled queue.

18. The method of claim 10 , wherein the job request is generated responsive to detecting a trigger condition associated with the action.

19. A non-transitory, computer-readable medium storing computer-executable instructions that, when executed by one or more processors, are to cause the one or more processors to:
receive a job request to implement one or more actions, the job request associated with a particular user;
compare a count of active job requests being currently processed for the particular user with an active job cap number for that user; and
selectively process the job request based on the comparison of the count of active job requests currently processed for the particular user with the active job cap number for that user,
wherein when the count of active job requests being currently processed for the particular user does not exceed the active job cap number, the job request is added to an active job queue for processing, and
wherein when the count of active job requests being currently processed for the particular user exceeds the active job cap number, the job request is placed on a throttled queue to await later processing when an updated count of active job requests being currently processed for the particular user is below

Assignees

• Shopify Inc

Inventors

• Mic Robert
• Manera Aline Fatima
• Willard Timothy
• Simone Nicole
• Weber Scott

Classifications

• G06F9/5005

  to service a request · CPC title

• G06F1/08

  Clock generators with changeable or programmable clock frequency · CPC title

• G06F9/54

  Interprogram communication · CPC title

• G06F2209/483

  Multiproc · CPC title

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title