Dynamically reducing stutter and latency in video streaming applications

Patent metadata

Field	Value
Publication number	US-12081812-B2
Application number	US-202318328854-A
Country	US
Kind code	B2
Filing date	Jun 5, 2023
Priority date	Mar 25, 2022
Publication date	Sep 3, 2024
Grant date	Sep 3, 2024

Abstract

A performance metrics of a receiver is obtained using frames of an application hosted by a server that are received via a network. The one or more performance metrics include information indicative of a current occupancy of a frame buffer corresponding to the receiver and information indicative of a target occupancy of the frame buffer corresponding to the receiver. The frame buffer of the receiver is used to queue frames of the application for display. A frame rate associated with rendering at least one next frame of the application is adjusted using the one or more performance metrics of the receiver to control population of the frame buffer. Subsequent frames of the application hosted by the server are rendered using the adjusted frame rate. Upon rendering the subsequent frames, the server sends the subsequent frames to the receiver for display.

First claim

What is claimed is:

1. A method comprising:
rendering a subsequent frame of an application using a frame rate associated with a population rate corresponding to a frame buffer of a receiver, the population rate being based at least on one or more of a current occupancy or a target occupancy of the frame buffer; and
transmitting, using a network, the rendered subsequent frame to the receiver to be displayed.

2. The method of claim 1 , wherein the target occupancy of the frame buffer is determined using at least one of: a moving average of a transmission duration or a moving average of a presentation duration.

3. The method of claim 2 , wherein the transmission duration is based at least on a difference between an expected arrival time of a rendered frame by the receiver and an actual arrival time of the rendered frame by the receiver.

4. The method of claim 2 , wherein the presentation duration is based at least on a period of time between populating the frame buffer with a frame and causing presentation of the frame on the receiver.

5. The method of claim 1 , wherein the current occupancy is determined based at least on a number of rendered frames of the application currently in the frame buffer.

6. The method of claim 1 , wherein the rendered subsequent frame is rendered using the frame rate, and further wherein the frame rate depends on a delay applied to an execution stage of a plurality of execution stages associated with the rendering of the rendered subsequent frame.

7. The method of claim 6 , wherein the delay is increased to decrease the frame rate and decreased to increase the frame rate.

8. A system comprising:
one or more processing units to:
render a subsequent frame of an application using a frame rate associated with a population rate corresponding to a frame buffer of a receiver, the population rate being based at least on one or more of a current occupancy or a target occupancy of the frame buffer; and
send, using a network, the subsequent frame to the receiver to cause a presentation of the subsequent frame.

9. The system of claim 8 , wherein the target occupancy of the frame buffer is determined using at least one of: a moving average of a transmission duration or a moving average of a presentation duration.

10. The system of claim 9 , wherein the transmission duration is based at least on a difference between an expected arrival time of a rendered frame by the receiver and an actual arrival time of the rendered frame by the receiver.

11. The system of claim 9 , wherein the presentation duration is based at least on a period of time between populating the frame buffer with a frame and causing presentation of the frame on the receiver.

12. The system of claim 8 , wherein the current occupancy is determined based at least on a number of rendered frames of the application currently in the frame buffer.

13. The system of claim 8 , wherein the subsequent frame is rendered using the frame rate, and further wherein the frame rate is determined based at least on a delay applied to an execution stage of a plurality of execution stages associated with the subsequent frame being rendered.

14. The system of claim 13 , wherein the delay is increased to decrease the frame rate and decreased to increase the frame rate.

15. One or more processing units comprising processing circuitry to cause performance of operations comprising:
rendering a subsequent frame of an application using a frame rate associated with a population rate corresponding to a frame buffer of a receiver, the population rate being based at least on one or more of a current occupancy or a target occupancy of the frame buffer; and
transmitting, using a network, the rendered subsequent frame to the receiver to be displayed.

16. The one or more processing units of claim 15 , wherein the target occupancy of the frame buffer is determined using at least one of: a moving average of a transmission duration or a moving average of a presentation duration.

17. The one or more processing units of claim 16 , wherein the transmission duration is based at least on a difference between an expected arrival time of a rendered frame by the receiver and an actual arrival time of the rendered frame by the receiver.

18. The one or more processing units of claim 16 , wherein the presentation duration is based at least on a period of time between populating the frame buffer with a frame and presenting the frame on the receiver.

19. The one or more processing units of claim 15 , wherein the current occupancy is determined based at least on a number of rendered frames of the application currently in the frame buffer.

20. The one or more processing units of claim 15 , wherein the rendered subsequent frame is rendered using the frame rate, and further wherein the frame rate depends on a delay applied to an execution stage of a plurality of execution stages associated with rendering the rendered subsequent frame, the delay being increased to decrease the frame rate and decreased to increase the frame rate.

Assignees

• Nvidia Corp

Inventors

• Dimitrov Rouslan
• Vandanov Viktor Grigoryevich
• Li Sau Yan Keith
• Howard James
• Cutler Scott Phillip

Classifications

• H04N21/2401

  Monitoring of the client buffer · CPC title

• A63F13/355

  Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client · CPC title

• H04L65/1046

  Call controllers; Call servers · CPC title

• H04L65/612

  for unicast · CPC title

• H04L47/25

  with rate being modified by the source upon detecting a change of network conditions · CPC title