Buffer drain rate tuning to a measured maximum receive bandwidth measured for a client device when streaming

Patent metadata

Field	Value
Publication number	US-12115447-B2
Application number	US-202218080605-A
Country	US
Kind code	B2
Filing date	Dec 13, 2022
Priority date	Oct 1, 2019
Publication date	Oct 15, 2024
Grant date	Oct 15, 2024

Abstract

A method for cloud gaming. The method including generating a plurality of video frames when executing a video game at a cloud gaming server. The method including encoding the plurality of video frames at an encoder bit rate, wherein the plurality of video frames that is compressed is transmitted to a client from a streamer of the cloud gaming server. The method including measuring a maximum receive bandwidth of a client. The method including monitoring the encoding of the plurality of video frames at the streamer. The method including dynamically tuning a parameter of the encoder based on the monitoring of the encoding.

First claim

What is claimed is:

1. A method for cloud gaming, comprising:
generating a plurality of video frames when executing a video game at a cloud gaming server;
measuring a maximum receive bandwidth of a client device;
encoding the plurality of video frames;
filling the plurality of video frames that is encoded to a buffer;
dynamically tuning a buffer drain rate to be below the maximum receive bandwidth, wherein the plurality of video frames that is encoded is drained from the buffer at the buffer drain rate for transmission to the client device over a network.

2. The method of claim 1 , wherein the dynamically tuning the buffer drain rate includes:
dynamically tuning the buffer drain rate to the maximum receive bandwidth.

3. The method of claim 1 , further comprising:
dynamically tuning an encoder bit rate to the maximum receive bandwidth; and
performing the encoding of the plurality of video frames at the encoder bit rate.

4. The method of claim 1 , further comprising:
filling the plurality of video frames that is encoded to the buffer as quickly as possible.

5. The method of claim 1 ,
wherein the maximum receive bandwidth is measured as a bit rate.

6. The method of claim 1 , further comprising:
modifying at least one of the plurality of video frames to generate a modified video frame by compositing the at least one of the plurality of video frames with a user interface feature; and
draining the modified video frame from the buffer at the buffer drain rate.

7. The method of claim 1 , wherein the measuring the maximum receive bandwidth includes:
draining a plurality of test packets from the buffer at a predefined bit rate that is a tested maximum receive bandwidth;
receiving feedback including a total number of the plurality of test packets received for a range of video frames;
determining a packet loss based on the total number of the plurality of test packets received and a number of the plurality of test packets; and
determining the maximum receive bandwidth based on the packet loss and the tested maximum receive bandwidth.

8. A non-transitory computer-readable medium storing a computer program for cloud gaming, the computer-readable medium comprising:
program instructions for generating a plurality of video frames when executing a video game at a cloud gaming server;
program instructions for measuring a maximum receive bandwidth of a client device;
program instructions for encoding the plurality of video frames;
program instructions for filling the plurality of video frames that is encoded to a buffer;
program instructions for dynamically tuning a buffer drain rate to be below the maximum receive bandwidth, wherein the plurality of video frames that is encoded is drained from the buffer at the buffer drain rate for transmission to the client device over a network.

9. The non-transitory computer-readable medium of claim 8 , wherein the program instructions for dynamically tuning the buffer drain rate includes:
program instructions for dynamically tuning the buffer drain rate to the maximum receive bandwidth.

10. The non-transitory computer-readable medium of claim 8 , further comprising:
program instructions for dynamically tuning an encoder bit rate to the maximum receive bandwidth; and
program instructions for performing the encoding of the plurality of video frames at the encoder bit rate.

11. The non-transitory computer-readable medium of claim 8 , further comprising:
program instructions for filling the plurality of video frames that is encoded to the buffer as quickly as possible.

12. The method of claim 1 ,
wherein in the method the maximum receive bandwidth is measured as a bit rate.

13. The non-transitory computer-readable medium of claim 8 , further comprising:
program instructions for modifying at least one of the plurality of video frames to generate a modified video frame by compositing the at least one of the plurality of video frames with a user interface feature; and
program instructions for draining the modified video frame from the buffer at the buffer drain rate.

14. The non-transitory computer-readable medium of claim 8 , wherein the program instructions for measuring the maximum receive bandwidth includes:
program instructions for draining a plurality of test packets from the buffer at a predefined bit rate that is a tested maximum receive bandwidth;
program instructions for receiving feedback including a total number of the plurality of test packets received for a range of video frames;
program instructions for determining a packet loss based on the total number of the plurality of test packets received and a number of the plurality of test packets; and
program instructions for determining the maximum receive bandwidth based on the packet loss and the tested maximum receive bandwidth.

15. A computer system comprising:
a processor; and
memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method for cloud gaming comprising:
generating a plurality of video frames when executing a video game at a cloud gaming server;
measuring a maximum receive bandwidth of a client device;
encoding the plurality of video frames;
filling the plurality of video frames that is encoded to a buffer;
dynamically tuning a buffer drain rate to be below the maximum receive bandwidth, wherein the plurality of video frames that is encoded is drained from the buffer at the buffer drain rate for transmission to the client device over a network.

16. The computer system of claim 15 , wherein in the method the dynamically tuning the buffer drain rate includes:
dynamically tuning the buffer drain rate to the maximum receive bandwidth.

17. The computer system of claim 15 , the method further comprising:
dynamically tuning an encoder bit rate to the maximum receive bandwidth; and
performing the encoding of the plurality of video frames at the encoder bit rate.

18. The computer system of claim 15 , the method further comprising:
filling the plurality of video frames that is encoded to the buffer as quickly as possible.

19. The computer system of claim 15 ,
wherein in the method the maximum receive bandwidth is measured as a bit rate.

20. The computer system of claim 15 , the method further comprising:
modifying at least one of the plurality of video frames to generate a modified video frame by compositing the at least one of the plurality of video frames with a user interface feature; and
draining the modified video frame from the buffer at the buffer drain rate.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Cerny Mark E
• Yong Kelvin M

Classifications

• A63F13/77

  involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory · CPC title

• A63F13/53

  involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game · CPC title

• H04N19/142

  Detection of scene cut or scene change · CPC title

• A63F2300/534

  for network load management, e.g. bandwidth optimization, latency reduction · CPC title

• H04N19/177

  the unit being a group of pictures [GOP] · CPC title