User-defined metered priority queues

Patent metadata

Field	Value
Publication number	US-11954518-B2
Application number	US-201916722422-A
Country	US
Kind code	B2
Filing date	Dec 20, 2019
Priority date	Dec 20, 2019
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

Apparatuses, systems, and techniques to optimize processor resources at a user-defined level. In at least one embodiment, priority of one or more tasks are adjusted to prevent one or more other dependent tasks from entering an idle state due to lack of resources to consume.

First claim

What is claimed is:

1. A method executed by a processor in a networked computing resource environment, comprising:
executing a high priority task in a priority queue of the processor, wherein the priority queue includes requests for a processing time of the processor, the requests including the plurality of tasks;
executing at least one feeder task in the priority queue, the at least one feeder task producing one or more intermediate outputs for consumption by the high priority task, wherein the high priority task is allocated, a larger amount of processing time of the processor than the at least one feeder task;
executing a synchronization mechanism indicating that at least one work queue is filled with the one or more intermediate outputs, and that the high priority task is consuming more of the one or more intermediate outputs than the one or more intermediate outputs the at least one feeder task is producing;
wherein
a priority for at least the at least one feeder task in the priority queue is adjusted to decrease an amount of switching between at least the at least one feeder task and the high priority task in the at least one work queue;

the processing time is reallocated in the priority queue based in part on at least the adjusted priority for the at least one feeder task in the priority queue to reduce a ratio of consumption of the one or more intermediate outputs to the production of the one or more intermediate outputs in the at least one work queue; and
executing the plurality of tasks according to the allocated reallocated processing time in the priority queue.

2. The method of claim 1 , wherein reducing the ratio includes increasing a priority of the at least one feeder task in the priority queue.

3. The method of claim 1 , wherein reducing the ratio includes decreasing a priority of the high priority task in the priority queue.

4. The method of claim 1 , wherein the high priority task and at least one feeder task are part of a deep learning pipeline.

5. The method of claim 4 , wherein the at least one feeder task includes receiving image data and processing the image data.

6. The method of claim 5 , wherein the high priority task provides output to a deep neural network and the processed image data is utilized as data for training the deep neural network.

7. The method of claim 1 , wherein an intermediate output of the one or more intermediate outputs is stored in the at least one work queue.

8. The method of claim 1 , wherein the priority queue is associated with one or more additional processors that receive requests including the plurality of tasks.

9. A method executed by a processor in a networked computing resource environment, comprising:
executing a first task, wherein work completed by the first task is provided to at least one work queue as one or more intermediate outputs;
executing a second task, wherein the second task consumes the one or more intermediate outputs from the at least one work queue, wherein the first task and the second task are executed according to requests for processing time in a priority queue of the processor;
executing a synchronization mechanism indicating that the at least one work queue is filled with the one or more intermediate outputs, and that the at least one work queue is being emptied of the one or more intermediate outputs faster than the one or more intermediate outputs are being provided to the at least one work queue;
wherein a priority of at least the first task and in the priority queue is adjusted to decrease an amount of switching between at least the first task and the second task in the at least one work queue, the adjusting resulting in reallocating the processing time in the priority queue for the first task and the second task; and
executing the first task and the second task according to the reallocated processing time in the priority queue.

10. The method of claim 9 , wherein:
wherein executing the synchronization mechanism further indicates whether the at least one work queue has available resources to receive additional intermediate output; and
adjusting the priority of the first task includes increasing an amount of intermediate output provided by the first task to the at least one work queue.

11. The method of claim 10 , wherein a result of executing the synchronization mechanism is a binary representation that indicates whether the at least one work queue has available space for more intermediate output.

12. The method of claim 9 , further including:
executing a third task, wherein the third task provides output to the at least one work queue as intermediate output, wherein a priority of the third task is adjusted to alter a rate at which intermediate work is being added to the at least one work queue.

13. The method of claim 9 , further comprising:
receiving input, wherein the input is provided to the first task for processing, wherein a final output is produced in response to completing processing of the one or more intermediate outputs produced by the second task.

14. The method of claim 13 , wherein the input is image data, wherein the first task processes the image data, and wherein the second task provides the final output to train a deep neural network based on the processed image data.

15. A system, comprising:
a processor in a networked computing resource environment for executing a plurality of tasks, the processor to:
execute a high priority task, wherein the high priority task is allocated a larger amount of processing time of the processor than one or more other tasks, and wherein one or more of the requests in a priority queue are related to the high priority task;
execute a feeder task, wherein the feeder task produces one or more intermediate outputs for consumption by the high priority task, wherein the feeder task is a lower priority in the priority queue than the high priority task, and wherein one or more of the requests in the priority queue are related to the feeder task,

wherein the requests sent to the priority queue of the processor by the feeder task and the high priority task are adjusted based, at least in part, on (i) execution of a synchronization mechanism indicating that at least one work queue is filled with the one or more intermediate outputs, and that the high priority task is consuming more of the one or more intermediate outputs than the feeder task is producing the one or more intermediate outputs, and ii in response to a priority being adjusted for at least one of the feeder task and the high priority task in the priority queue to decrease an amount of switching between the feeder task and the high priority task in the at least one work queue; and
execute the feeder task and the high priority task according to the adjusted requests sent to the priority queue.

16. The system of claim 15 , wherein:
the at least one work queue includes intermediate output produced by the feeder task and not yet consumed by the high priority task.

17. The system of claim 15 , wherein the processor is further to:
receive input, wherein the input is provided to the feeder task for processing, wherein a final output is produced in response to completing processing of the one or more intermediate outputs by the second task.

18. The system of claim 17 , wherein the input is image data, wherein the feeder task processes the image data, and wherein the high priority task provides the final output to train a deep neural network based on the processed image data.

19. The system of claim 15 , wherein the processor is furthe

Assignees

• Nvidia Corp

Inventors

• Evans Jonathon
• Shah Lacky
• Johnson Phil
• Alben Jonah
• Pharris Brian
• Palmer Greg
• Fahs Brian

Classifications

• G06F9/4831 Primary

  with variable priority · CPC title

• G06N3/08

  Learning methods · CPC title

• G06F9/5038 Primary

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F2209/5021

  Priority · CPC title