Offloading stream processing tasks to parallel processing units for content streaming systems and applications

Patent metadata

Field	Value
Publication number	US-12506909-B2
Application number	US-202318331757-A
Country	US
Kind code	B2
Filing date	Jun 8, 2023
Priority date	Jun 8, 2023
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

In various examples, processing content data using graphics processing units for video streaming systems and applications is described herein. Systems and methods are disclosed that offload at least a portion of the processing that is typically performed by a central processing unit (CPU) to a graphics processing unit (GPU). For example, and for a streaming application, the GPU may initially generate and then encode a content stream (e.g., a video stream, an audio stream, etc.). In some examples, the GPU generates and/or encodes the content stream based on input data received from one or more client devices. The GPU may then perform additional processing associated with the encoded content stream, such as packetization, forward error correction (FEC), encryption, and/or any other processing. The CPU may then use packet pacing when causing the processed content stream to be streamed to the client device(s).

First claim

What is claimed is:

1 . A method comprising:
generating, using one or more parallel processing units, video data associated with an application; generating, using the one or more parallel processing units, encrypted video data by encrypting the video data; generating, using one or more central processing units, encrypted audio data by encrypting audio data associated with the application; receiving, using the one or more central processing units, the encrypted video data from the one or more parallel processing units; and causing, using the one or more central processing units, the encrypted video data and the encrypted audio data to be streamed to one or more client devices.

2 . The method of claim 1 , wherein the generating the video data, the generating the encrypted video data, and the generating of the encrypted audio data is performed by one or more remote systems that communicate with the one or more client devices via one or more networks.

3 . The method of claim 1 , further comprising:
generating, using the one or more parallel processing units, processed video data by processing the video data using forward error correction, wherein the generating of the encrypted video data includes encrypting the processed video data.

4 . The method of claim 1 , further comprising:
packetizing, using the one or more parallel processing units, at least a portion of the video data as a plurality of data packets, wherein the generating the encrypted video data comprises generating, using the one or more parallel processing units, a plurality of encrypted data packets by encrypting the plurality of data packets.

5 . The method of claim 1 , further comprising:
generating, using the one or more parallel processing units, encoded video data by encoding the video data, wherein the generating the encrypted video data includes encrypting the encoded video data.

6 . The method of claim 1 , wherein the causing the encrypted video data and the encrypted audio data to be streamed to the one or more client devices comprises:
processing, using the one or more central processing units, the encrypted video data and the encrypted audio data using packet pacing; and sending, using one or more network interfaces and based at least on the packet pacing, the encrypted video data and the encrypted audio data to the one or more client devices.

7 . The method of claim 1 , wherein the generating the encrypted audio data using the one or more central processing units occurs at least partially in parallel with the generating the encrypted video data using the one or more parallel processing units.

8 . The method of claim 1 , further comprising:
receiving an encryption key, wherein:
the generating the encrypted video data is based at least on the encryption key; and
the generating the encrypted audio data is based at least on the encryption key.

9 . The method of claim 1 , further comprising:
receiving, from the one or more client devices, input data representative of one or more inputs, wherein the generating the video data is based at least on the input data.

10 . A system comprising:
one or more graphics processing units to:
generate a first video stream associated with an application;
generate a second video stream by encrypting the first video stream; and
cause the second video stream to be provided to one or more central processing units; and
the one or more central processing units to:
obtain a first audio stream associated with the application;
generate a second audio stream by encrypting the first audio stream; and
stream the second video stream and the second audio stream to one or more client devices.

11 . The system of claim 10 , wherein:
the one or more graphics processing units are further to generate a third video stream by processing the first video stream using forward error correction; and the generation of the second video stream includes encrypting the third video stream.

12 . The system of claim 10 , wherein:
the one or more graphics processing units are further to generate, based at least on the first video stream, a plurality of first data packets; and the generation of the second video stream comprises generating, using the one or more graphics processing units, a plurality of second data packets by encrypting the plurality of first data packets.

13 . The system of claim 10 , wherein:
the one or more graphics processing units are further to generate a third video stream by encoding the first video stream using one or more network protocols; and the generation of the second video stream includes encrypting the third video stream.

14 . The system of claim 10 , wherein:
the one or more central processing units are further to process the second video stream and the second audio stream using packet pacing; and the second video stream and the second audio stream are streamed based at least on the packet pacing.

15 . The system of claim 10 , wherein:
the generation of the second video stream is based at least on an encryption key; and the generation the second audio stream is based at least on the encryption key.

16 . The system of claim 10 , wherein:
the one or more graphics processing units are further to receive input data representative of one or more inputs, the input data generated using the one or more client devices; and the generation of the first video stream is based at least on the input data.

17 . The system of claim 10 , wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

18 . The system of claim 10 , wherein the generation of the second video stream using the one or more central processing units occurs at least partially in parallel with the generation of the second video data using the one or more graphics processing units.

19 . One or more processors comprising:
one or more graphics processing units to:
generate a processed video stream by least processing a video stream using encryption and forward error correction, wherein generation of the video stream is based at least on input data received from one or more client devices; and
one or more central processing units to:
generate a processed audio stream by at least processing an audio stream using encryption; and
cause the processed video stream and the processed audio stream to be streamed to the one or more client devices.

20 . The one or more processors of claim 19 , wherein the one or more processors are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation oper

Assignees

• Nvidia Corp

Inventors

• Parikh Amit
• Kasi Ganapathy Raman
• Lapicque Olivier

Classifications

• H04L63/0457

  wherein the sending and receiving network entities apply dynamic encryption, e.g. stream encryption (cryptographic mechanisms or cryptographic arrangements for stream encryption H04L9/065) · CPC title

• H04L63/0428

  wherein the data content is protected, e.g. by encrypting or encapsulating the payload · CPC title

• H04N21/2347 Primary

  involving video stream encryption · CPC title