Encoding output for streaming applications based on client upscaling capabilities

Patent metadata

Field	Value
Publication number	US-2025184383-A1
Application number	US-202519044425-A
Country	US
Kind code	A1
Filing date	Feb 3, 2025
Priority date	Feb 28, 2022
Publication date	Jun 5, 2025
Grant date	—

Abstract

In various examples, the decoding and upscaling capabilities of a client device are analyzed to determine encoding parameters and operations used by a content streaming server to generate encoded video streams. The quality of the upscaled content of the client device may be monitored by the streaming servers such that the encoding parameters may be updated based on the monitored quality. In this way, the encoding operations of one or more streaming servers may be more effectively matched to the decoding and upscaling abilities of one or more client devise such that an increased number of client devices may be served by the streaming servers.

First claim

What is claimed is:

1 . A method comprising:
obtaining data indicating one or more upscaling ratios associated with one or more upscaling algorithms available to a client device; selecting a resolution for one or more streams of video data to the client device based at least on the one or more upscaling ratios; encoding one or more portions of the one or more streams of video data using the resolution based at least on the selection; and transmitting the encoded one or more portions of the one or more streams to the client device, the transmitting causing the client device to upscale one or more video frames represented by the video data using an upscaling algorithm of the one or more upscaling algorithms.

2 . The method of claim 1 , further comprising:
identifying, using the data, at least one upscaling algorithm of the one or more upscaling algorithms; and based at least on the identifying, determining at least one upscaling ratio of the one or more upscaling ratios, wherein the resolution is selected based at least on the at least one upscaling ratio.

3 . The method of claim 1 , further comprising:
determining, using the data, a set of upscaling ratios based at least on an indication that the set of upscaling ratios are compatible with at least one upscaling algorithm of the one or more upscaling algorithms; and based at least on the determining, selecting, for the one or more streams, the one or more upscaling ratios as a subset of the set of upscaling ratios.

4 . The method of claim 1 , wherein the selecting is based at least on the one or more upscaling ratios being capable of upscaling the resolution to one or more of:
a current resolution being used by the client device to present the one or more streams, or one or more supported resolutions that are usable by the client device to present the one or more streams.

5 . The method of claim 1 , wherein the selecting is based at least on the resolution being less than or equal to a maximum resolution that is supported by one or more of:
at least one upscaling algorithm of the one or more upscaling algorithms, or one or more displays of the client device.

6 . The method of claim 1 , wherein the selecting is based at least on the resolution being greater than or equal to a minimum resolution that is supported by at least one upscaling algorithm of the one or more upscaling algorithms.

7 . The method of claim 1 , wherein the transmitting further causes the client device to switch from using a second upscaling algorithm to present the one or more streams to using the upscaling algorithm to present the one or more streams.

8 . The method of claim 1 , wherein the selecting is based at least on the resolution being greater than or equal to a minimum resolution that is selected by the client device.

9 . The method of claim 1 , wherein the selecting of the resolution is further based at least on the data indicating the one or more upscaling algorithms include at least one artificial intelligence-based upscaling algorithm.

10 . The method of claim 1 , wherein the video data comprises game content from one or more instances of a game application, the game content is included in the one or more streams, and the method further includes applying input from the client device to at least one instance of the one or more instances of the game application.

11 . A system comprising:
one or more processors to execute operations including:
obtaining data indicating one or more upscaling ratios associated with one or more upscaling algorithms available to a client device;
selecting a resolution for one or more streams of video data to the client device based at least on the one or more upscaling ratios;
encoding one or more portions of the one or more streams of video data using the resolution based at least on the selection; and
transmitting the encoded one or more portions of the one or more streams to the client device, the transmitting causing the client device to upscale one or more video frames represented by the video data using an upscaling algorithm of the one or more upscaling algorithms.

12 . The system of claim 11 , wherein the operations further include:
identifying, using the data, at least one upscaling algorithm of the one or more upscaling algorithms; and based at least on the identifying, determining at least one upscaling ratio of the one or more upscaling ratios, wherein the resolution is selected based at least on the at least one upscaling ratio.

13 . The system of claim 11 , wherein the operations further include:
determining, using the data, a set of upscaling ratios based at least on an indication that the set of upscaling ratios are compatible with at least one upscaling algorithm of the one or more upscaling algorithms; and based at least on the determining, selecting, for the one or more streams, the one or more upscaling ratios as a subset of the set of upscaling ratios.

14 . The system of claim 11 , wherein the selecting is based at least on the one or more upscaling ratios being capable of upscaling the resolution to one or more of:
a current resolution being used by the client device to present the one or more streams, or one or more supported resolutions that are usable by the client device to present the one or more streams.

15 . The system of claim 11 , wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

16 . At least one processor comprising:
one or more circuits to cause a client device to upscale one or more video frames represented by video data of one or more streams using an upscaling algorithm of one or more upscaling algorithms available to the client device based at least on:
receiving data indicating one or more upscaling ratios associated with the one or more upscaling algorithms;
selecting a resolution for the one or more streams based at least on the one or more upscaling ratios; and
encoding one or more portions of the one or more streams of the video data using the resolution based at least on the selection.

17 . The at least one processor of claim 16 , the one or more circuits are further to cause the client device to upscale the one or more video frames based at least on:
identifying, using the data, at least one upscaling algorithm of the one or more upscaling algorithms; and based at least on the identifying, determining at least one upscaling ratio of the one or more upscaling ratios, wherein the resolution is selected based at least on the at least one upscaling ratio.

18 . The at least one processor of claim 16 , the one or more circuits are further to cause the client device to upscale the one or more video frames based at least on:
determining, using the data, a set of upscaling ratios based at least on an indication that the set of upscaling ratios are compatible w

Assignees

• Nvidia Corp

Inventors

• Sundareson Prabindh
• Pandhare Sachin
• Raikar Shyam

Classifications

• H04N19/59

  involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution · CPC title

• H04N19/146

  Data rate or code amount at the encoder output · CPC title

• H04N19/105

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04L65/762

  at the source  (reformatting of additional data in video distribution servers H04N21/2355) · CPC title

• H04L65/764

  at the destination  (reformatting of additional data in video clients H04N21/4355) · CPC title