Software-assisted instruction level execution preemption

Patent metadata

Field	Value
Publication number	US-10552201-B2
Application number	US-201715594515-A
Country	US
Kind code	B2
Filing date	May 12, 2017
Priority date	Nov 8, 2011
Publication date	Feb 4, 2020
Grant date	Feb 4, 2020

Abstract

One embodiment of the present invention sets forth a technique for instruction level execution preemption. Preempting at the instruction level does not require any draining of the processing pipeline. No new instructions are issued and the context state is unloaded from the processing pipeline. Any in-flight instructions that follow the preemption command in the processing pipeline are captured and stored in a processing task buffer to be reissued when the preempted program is resumed. The processing task buffer is designated as a high priority task to ensure the preempted instructions are reissued before any new instructions for the preempted context when execution of the preempted context is restored.

First claim

The invention claimed is:

1. A method of preempting execution of program instructions in a multi-threaded system, the method comprising:
executing program instructions in a processing pipeline within the multi-threaded system using a first context;
receiving a first preempt command for initiating preempting execution using the first context to execute different program instructions in the multi-threaded system using a second context;
specifying, for the first context, a first preemption technique at an instruction level based on a length of time for executing the first context;
in response to receiving the first preempt command, implementing the first preemption technique comprising:
determining that the processing pipeline is not idle;
in response to determining that the processing pipeline is not idle, stopping execution of instructions for the first context;
storing an indication that execution of the program instructions using the first context was preempted; and
executing the different program instructions in the processing pipeline using the second context.

2. The method of claim 1 , further comprising, before executing the different program instructions, storing a portion of first context state that is maintained within the processing pipeline during execution of the program instructions using the first context.

3. The method of claim 2 , further comprising preempting execution of the different program instructions to resume execution of the program instructions using the portion of the first context state.

4. The method of claim 2 , further comprising:
restoring the portion of the first context state to the processing pipeline; and
resuming execution of the program instructions using the portion of the first context state.

5. The method of claim 1 , wherein the preempting of the execution using the first context further comprises storing first context state for each thread group executing in a streaming multiprocessor that is preempted when preemption occurs at the instruction level.

6. The method of claim 5 , further comprising storing additional first context state for each compute thread array that is preempted when preemption occurs at the instruction level.

7. The method of claim 1 , wherein the preempting of the execution using the first context further comprises determining that streaming multiprocessors configured to execute the program instructions using the first context are idle.

8. The method of claim 1 , further comprising:
determining, before executing the different program instructions, that the processing pipeline is idle; and
resetting the processing pipeline without storing context state maintained in the processing pipeline for the first context.

9. The method of claim 1 , further comprising storing first context state indicating that a wait-for-idle command was being executed for the first context.

10. The method of claim 1 , wherein preempting execution using the first context at an instruction level comprises stopping execution of the program instructions at an instruction boundary of a streaming multiprocessor configured to execute the program instructions.

11. The method of claim 1 , wherein:
the first context has a longer execution time than the second context; and
the first preemption technique for preempting execution using the first context is different than a second preemption technique for preempting execution using the second context.

12. The method of claim 1 , wherein the first context has a longer execution time than the second context, the method further comprising:
executing program instructions in the processing pipeline using the second context;
receiving a second preempt command for initiating preempting execution using the second context; and
in response to receiving the second preempt command, preempting execution using the second context by implementing a second preemption technique that is different than the first preemption technique.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to preempt execution of program instructions in a multi-threaded system, by performing the steps of:
executing program instructions in a processing pipeline within the multi-threaded system using a first context;
receiving a first preempt command for initiating preempting execution using the first context to execute different program instructions in the multi-threaded system using a second context;
specifying, for the first context, a first preemption technique at an instruction level based on a length of time for executing the first context;
in response to receiving the first preempt command, implementing the first preemption technique comprising:
determining that the processing pipeline is not idle;
in response to determining that the processing pipeline is not idle, stopping execution of instructions for the first context;
storing an indication that execution of the program instructions using the first context was preempted; and
execute the different program instructions in the processing pipeline using the second context.

14. A multi-threaded system for preempting execution of program instructions, the multi-threaded system comprising:
a memory configured to store program instruction corresponding to a first context and different program instructions corresponding to a second context;
a host interface coupled to a processing pipeline and configured to preempt execution of the program instructions using the first context; and
the processing pipeline configured to:
execute the program instructions using the first context;
receive a first preempt command for initiating preempting execution using the first context to execute different program instructions in the multi-threaded system using the second context;
specify, for the first context, a first preemption technique at an instruction level based on a length of time for executing the first context;
in response to receiving the first preempt command, implement the first preemption technique comprising:
determine that the processing pipeline is not idle;
in response to determining that the processing pipeline is not idle, stop execution of instructions for the first context;
store an indication that execution of the program instructions using the first context was preempted; and
execute the different program instructions using the second context.

15. The multi-threaded system of claim 14 , wherein the processing pipeline is further configured to, before executing the different program instructions, store a portion of first context state that is maintained within the processing pipeline during the execution of the program instructions using the first context.

16. The multi-threaded system of claim 15 , wherein the host interface is further configured to preempt execution of the different program instructions to resume execution of the program instructions using the portion of the first context state.

17. The multi-threaded system of claim 15 , wherein the processing pipeline is further configured to:
restore the portion of the first context state; and
resume execution of the program instructions using the portion of the first context state.

18. The multi-threaded system of claim 14 , wherein the processing pipeline is further configured to store first context state for each thread group executing in a streaming multiprocessor that is preempted in the memory when preemption occurs at the instruction level.

19. The multi-th

Assignees

• Nvidia Corp

Inventors

• Cuadra Philip Alexander
• Lamb Christopher
• Shah Lacky V

Classifications

• G06F9/461 Primary

  Saving or restoring of program or task context · CPC title

• G06F9/3848

  using hybrid branch prediction, e.g. selection between prediction techniques · CPC title

• G06F9/3867

  using instruction pipelines · CPC title

• G06F9/3861

  Recovery, e.g. branch miss-prediction, exception handling (error detection or correction G06F11/00) · CPC title

• G06F9/3842

  Speculative instruction execution · CPC title