Methods and apparatus for auto-throttling encapsulated compute tasks

Patent metadata

Field	Value
Publication number	US-9710306-B2
Application number	US-201213442730-A
Country	US
Kind code	B2
Filing date	Apr 9, 2012
Priority date	Apr 9, 2012
Publication date	Jul 18, 2017
Grant date	Jul 18, 2017

Abstract

Systems and methods for auto-throttling encapsulated compute tasks. A device driver may configure a parallel processor to execute compute tasks in a number of discrete throttled modes. The device driver may also allocate memory to a plurality of different processing units in a non-throttled mode. The device driver may also allocate memory to a subset of the plurality of processing units in each of the throttling modes. Data structures defined for each task include a flag that instructs the processing unit whether the task may be executed in the non-throttled mode or in the throttled mode. A work distribution unit monitors each of the tasks scheduled to run on the plurality of processing units and determines whether the processor should be configured to run in the throttled mode or in the non-throttled mode.

First claim

What is claimed is:

1. A method for auto-throttling encapsulated compute tasks, the method comprising:
receiving one or more tasks to be scheduled for execution by a processor that includes a plurality of parallel processing units, wherein the processor may be configured in a non-throttled mode in which a first number of the plurality of parallel processing units are active for processing tasks or a throttled mode in which a second number of the plurality of parallel processing units are active for processing tasks, the second number being smaller than the first number;
determining that at least one of the tasks scheduled for execution includes a throttled flag, wherein the throttled flag indicates that the corresponding task is to be executed only when the processor is configured in the throttled mode; and
in response to determining that the at least one of the tasks scheduled for execution includes the throttled flag, configuring the processor to activate the second number of parallel processing units.

2. The method of claim 1 , wherein the processor comprises a graphics processing unit (GPU) that includes a work distribution unit coupled to the plurality of parallel processing units and that schedules tasks for execution on the plurality of parallel processing units.

3. The method of claim 2 , wherein determining whether at least one of the tasks scheduled for execution may be executed only when the processor is configured in the throttled mode comprises:
storing the tasks scheduled for execution in a task table; and
performing an OR operation using a throttled flag associated with each task stored in the task table.

4. The method of claim 3 , wherein each task in the one or more tasks comprises a data structure stored in a memory that includes a field that represents the throttled flag.

5. The method of claim 4 , wherein the field comprises a single bit that is set to indicate that the task may be executed only when the processor is configured in the throttled mode or cleared to indicate that the task may be executed in either the throttled mode or the non-throttled mode.

6. The method of claim 4 , wherein the field comprises a multi-bit value that corresponds to three or more discrete throttling states, the multi-bit value indicating in which discrete throttling state the task may be executed, and wherein each discrete throttling state corresponds to a different number of active parallel processing units.

7. The method of claim 1 , further comprising:
if the processor is configured in the non-throttled mode, then associating each of the activated parallel processing units with a first range of virtual memory addresses, wherein each parallel processing unit is associated with a unique index and each thread that executes on a particular parallel processing unit is associated with a unique thread index; or
if the processor is configured in the throttled mode, then associating each of the activated parallel processing units with a second range of virtual memory addresses,
wherein the second range of virtual memory addresses is larger than the first range of virtual memory addresses for each of the parallel processing units activated in the throttled mode.

8. The method of claim 7 , wherein a device driver allocates a buffer in a memory coupled to the processor to provide a private local memory for each of the threads executing on the parallel processing units, and wherein a memory mapping unit in the processor translates virtual memory addresses into locations within the buffer.

9. The method of claim 8 , wherein each task is associated with a throttled flag, and wherein the device driver, upon determining that a size of local memory required by a particular task is above a threshold value, sets the throttled flag associated with the particular task.

10. A processor for auto-throttling encapsulated compute tasks, the processor comprising:
a plurality of parallel processing units, wherein the processor may be configured in a non-throttled mode in which a first number of the plurality of parallel processing units are active for processing tasks or a throttled mode in which a second number of the plurality of parallel processing units are active for processing tasks, the second number being smaller than the first number; and
a work distribution unit coupled to the plurality of parallel processing units, wherein the work distribution unit:
receives one or more tasks to be scheduled for execution,
determines that at least one of the tasks scheduled for execution includes a throttled flag, wherein the throttled flag indicates that the corresponding task is to be executed only when the processor is configured in the throttled mode, and
in response to determining that the at least one of the tasks scheduled for execution includes the throttled flag, configures the processor to activate the second number of parallel processing units.

11. The processor of claim 10 , wherein the processor comprises a graphics processing unit (GPU).

12. The processor of claim 11 , wherein determining whether at least one of the tasks scheduled for execution may be executed only when the processor is configured in the throttled mode comprises:
storing the tasks scheduled for execution in a task table; and
performing an OR operation using a throttled flag associated with each task stored in the task table.

13. The processor of claim 12 , wherein each task in the one or more tasks comprises a data structure stored in a memory that includes a field that represents the throttled flag.

14. The processor of claim 13 , wherein the field comprises a single bit that is set to indicate that the task may be executed only when the processor is configured in the throttled mode or cleared to indicate that the task may be executed in either the throttled mode or the non-throttled mode.

15. The processor of claim 13 , wherein the field comprises a multi-bit value that corresponds to three or more discrete throttling states, the multi-bit value indicating in which discrete throttling state the task may be executed, and wherein each discrete throttling state corresponds to a different number of active parallel processing units.

16. The processor of claim 10 , wherein a device driver:
associates each of the parallel processing units activated in the non-throttled mode with a first range of virtual memory addresses, wherein each parallel processing unit is associated with a unique index and each thread that executes on a particular parallel processing unit is associated with a unique thread index; or
associates each of the parallel processing units activated in the throttled mode with a second range of virtual memory addresses,
wherein the second range of virtual memory addresses is larger than the first range of virtual memory addresses for each of the parallel processing units activated in the throttled mode.

17. The processor of claim 16 , wherein the device driver allocates a buffer in a memory coupled to the processor to provide a private local memory for each of the threads executing on the parallel processing units, and wherein a memory mapping unit in the processor translates virtual memory addresses into locations within the buffer.

18. The processor of claim 17 , wherein each task is associated with a throttled flag, and wherein the device driver, upon determining that a size of local memory required by a particular task is above a threshold value, sets the throttled flag associated with the particular task.

19. The processor of claim 10 , wherein configuring t

Assignees

• Duluk Jr Jerome F
• Hall Jesse David
• Cuadra Philip Alexander
• Abdalla Karim M
• Nvidia Corp

Inventors

• Duluk Jr Jerome F
• Hall Jesse David
• Cuadra Philip Alexander
• Abdalla Karim M

Classifications

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F2209/483

  Multiproc · CPC title

• G06F9/5016

  the resource being the memory · CPC title