Dynamically reducing latency in processing pipelines

Patent metadata

Field	Value
Publication number	US-12437356-B2
Application number	US-202418594099-A
Country	US
Kind code	B2
Filing date	Mar 4, 2024
Priority date	Sep 21, 2021
Publication date	Oct 7, 2025
Grant date	Oct 7, 2025

Abstract

A weighted average execution time associated with each execution stage of a plurality of execution stages used to process a plurality of frames in parallel is obtained. The processing of each of the plurality of frames is performed at each of the plurality of execution stages in a sequential order, starting with an initial execution stage and continuing with each subsequent execution stage. A first largest weighted average execution time associated with one of the plurality of execution stages is determined. A delay to the initial execution stage prior to processing a first next frame is applied. The delay is determined based on the first largest weighted average execution time.

First claim

What is claimed is:

1. A method comprising:
obtaining a weighted average execution time associated with each execution stage of a plurality of execution stages processing a plurality of frames starting with an initial execution stage and continuing with each subsequent execution stage;
determining a first largest weighted average execution time associated with one of the plurality of execution stages; and
applying a delay to the initial execution stage prior to processing a first next frame of the plurality of frames, wherein the delay is based on the first largest weighted average execution time of the plurality of execution stages.

2. The method of claim 1 , further comprising:
updating, upon processing the first next frame in at least one execution stage of the plurality of execution stages, a weighted average execution time of the at least one execution stage of the plurality of execution stages;
determining a second largest weighted average execution time in view of the updated weighted average execution time of the at least one execution stage; and
updating the delay to be applied to the initial execution stage based on the second largest weighted average execution time.

3. The method of claim 1 , wherein the delay is equal to a difference between the first largest weighted average execution time and an execution time of a previous frame in the initial execution stage of the plurality of execution stages.

4. The method of claim 1 , further comprising:
adjusting, prior to processing subsequent frames, the delay to be applied to the initial execution stage based on a period of time between (i) processing of a second next frame at an execution stage preceding an execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time.

5. The method of claim 4 , wherein adjusting the delay comprises:
determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is greater than zero; and
increasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

6. The method of claim 4 , wherein adjusting the delay comprises:
determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is less than zero; and
decreasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

7. The method of claim 6 , wherein the predefined minimum value indicates there is no delay or queue between processing of the second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first subsequent frame at the execution stage associated with the first largest weighted average execution time processing.

8. The method of claim 4 , wherein adjusting the delay comprises:
receiving, using a proportional-integral-derivative (PID) controller, the period of time between processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first next frame at the execution stage associated with the first largest weighted average execution time; and
outputting, using the PID controller, an adjustment value to be applied to the delay based on the period of time matching a predefined minimum value; and
applying the adjustment value to the delay.

9. A system comprising:
one or more processing units to:
obtaining a weighted average execution time associated with each execution stage of a plurality of execution stages processing a plurality of frames starting with an initial execution stage and continuing with each subsequent execution stage;
determining a first largest weighted average execution time associated with one of the plurality of execution stages; and
applying a delay to the initial execution stage prior to processing a first next frame of the plurality of frames, wherein the delay is based on the first largest weighted average execution time of the plurality of execution stages.

10. The system of claim 9 , wherein the one or more processing units further:
update, upon processing the first next frame in at least one execution stage of the plurality of execution stages, a weighted average execution time of the at least one execution stage of the plurality of execution stages;
determine a second largest weighted average execution time in view of the updated weighted average execution time of the at least one execution stage; and
update the delay to be applied to the initial execution stage based on the second largest weighted average execution time.

11. The system of claim 9 , wherein the delay is equal to a difference between the first largest weighted average execution time and an execution time of a previous frame in the initial executions stage of the plurality of execution stages.

12. The system of claim 9 , wherein the one or more processing units further:
adjust, prior to processing subsequent frames, the delay to be applied to the initial execution stage based on a period of time between (i) processing of a second next frame at an execution stage preceding an execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time.

13. The system of claim 12 , wherein to adjust the delay, the one or more processing units further:
determine that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is greater than zero; and
increase the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

14. The system of claim 12 , wherein to adjust the delay, the one or more processing units further:
determine that the period of time between (i) processing of the second next frame at the execution stage pre

Assignees

• Nvidia Corp

Inventors

• Li Sau Yan Keith
• Schneider Seth
• Robson Cody
• Nordskog Lars
• Hansen Charles
• Dimitrov Rouslan

Classifications

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/3836

  Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution · CPC title

• G06F9/4887

  involving deadlines, e.g. rate based, periodic · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title