Congestion control for low-latency interactive video streaming

Patent metadata

Field	Value
Publication number	US-2025039099-A1
Application number	US-202418790035-A
Country	US
Kind code	A1
Filing date	Jul 31, 2024
Priority date	Apr 4, 2022
Publication date	Jan 30, 2025
Grant date	—

Abstract

A server transmits an encoded game frame over a network to a respective client system as a set of packets. In response to transmitting the set of packets, the server determines a bandwidth estimate based on the size of the encoded game frame and the timing data associated with the transmitted set of packets. The server then compares the bandwidth estimate to a current video bitrate of the game stream being transmitted from the server to the respective client device. In response to the comparison indicating an underutilization of the network, the server increases the encoding bitrate. Further, in response to the comparison indication an overutilization of the network, the server decreases the encoding bitrate.

First claim

1 - 20 . (canceled)

21 . A method, comprising:
based on encoding a frame according to an encoding bitrate to produce an encoded frame, transmitting the encoded frame as a set of packets over a network; determining a bandwidth estimate based on timing data associated with transmitting the set of packets over the network and based on an undershoot correction factor associated with a size of the encoded frame; and modifying the encoding bitrate based on the bandwidth estimate.

22 . The method of claim 21 , wherein the timing data comprises a first set of time stamps indicating when the set of packets was transmitted and a second set of time stamps indicating when the set of packets was received by a device.

23 . The method of claim 21 , wherein modifying the encoding bitrate comprises:
based on the bandwidth estimate being less than a current video bitrate, decreasing the encoding bitrate.

24 . The method of claim 21 , wherein modifying the encoding bitrate comprises:
based on the bandwidth estimate being greater than a current video bitrate, increasing the encoding bitrate.

25 . The method of claim 21 , wherein determining the bandwidth estimate comprises:
determining a time interval to transmit the encoded frame based on the timing data, wherein the bandwidth estimate is determined based on the time interval.

26 . The method of claim 21 , wherein determining the bandwidth estimate comprises:
modifying the bandwidth estimate based on a predetermined step size.

27 . The method of claim 21 , further comprising:
based on encoding a second frame to produce a second encoded frame, transmitting the second encoded frame as a second set of packets over the network according to a pacing rate based on the bandwidth estimate.

28 . A computing device, comprising:
an encoder configured to encode a frame according to an encoding bitrate to produce an encoded frame; and a processor configured to:
transmit the encoded frame as a set of packets over a network;
determine a bandwidth estimate based on timing data associated with transmitting the set of packets over the network and based on an undershoot correction factor associated with a size of the encoded frame; and
modify the encoding bitrate based on the bandwidth estimate.

29 . The computing device of claim 28 , wherein the timing data comprises a first set of time stamps indicating when the set of packets was transmitted and a second set of time stamps indicating when the set of packets was received by a device.

30 . The computing device of claim 28 , wherein the processor is configured to:
based on the bandwidth estimate being lower than a current video bitrate, decrease the encoding bitrate.

31 . The computing device of claim 28 , wherein the processor is configured to:
based on the bandwidth estimate being greater than a current video bitrate, increase the encoding bitrate.

32 . The computing device of claim 28 , wherein the processor is configured to:
determine a time interval to transmit the encoded frame based on the timing data, wherein the bandwidth estimate is determined based on the time interval.

33 . The computing device of claim 28 , wherein the processor is configured to:
modify the bandwidth estimate based on a predetermined step size.

34 . The computing device of claim 28 , wherein the processor is configured to:
based on the encoder encoding a second frame to produce a second encoded frame, transmit the second encoded frame as a second set of packets over the network according to a pacing rate based on the bandwidth estimate.

35 . A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing device, cause the at least one processor to:
encode a frame according to an encoding bitrate to produce an encoded frame; transmit the encoded frame as a set of packets over a network; determine a bandwidth estimate based on timing data associated with transmitting the set of packets over the network and based on an undershoot correction factor associated with a size of the encoded frame; and modify the encoding bitrate based on the bandwidth estimate.

36 . The non-transitory computer-readable storage medium of claim 35 , wherein the timing data comprises a first set of time stamps indicating when the set of packets was transmitted and a second set of time stamps indicating when the set of packets was received by a device.

37 . The non-transitory computer-readable storage medium of claim 35 further including instructions that cause the at least one processor to:
based on the bandwidth estimate being lower than a current video bitrate, decrease the encoding bitrate.

38 . The non-transitory computer-readable storage medium of claim 35 further including instructions that cause the at least one processor to:
based on the bandwidth estimate being greater than a current video bitrate, increase the encoding bitrate.

39 . The non-transitory computer-readable storage medium of claim 35 further including instructions that cause the at least one processor to:
modify the bandwidth estimate based on a predetermined step size.

40 . The non-transitory computer-readable storage medium of claim 35 further including instructions that cause the at least one processor to:
based on encoding a second frame to produce a second encoded frame, transmit the second encoded frame as a second set of packets over the network according to a pacing rate based on the bandwidth estimate.

Assignees

• Google Llc

Inventors

• Wei Teng
• Smith Connor
• Chu David
• Ray Devdeep
• Reinert Bernhard
• Zhang Zengbin

Classifications

• A63F13/67

  adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use · CPC title

• H04N21/4781

  Games · CPC title

• H04N21/2402

  Monitoring of the downstream path of the transmission network, e.g. bandwidth available (traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876) · CPC title

• H04N21/23805

  Controlling the feeding rate to the network, e.g. by controlling the video pump · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title