Video compression techniques for reliable transmission

Patent metadata

Field	Value
Publication number	US-12335458-B2
Application number	US-202117365669-A
Country	US
Kind code	B2
Filing date	Jul 30, 2021
Priority date	Jul 30, 2021
Publication date	Jun 17, 2025
Grant date	Jun 17, 2025

Abstract

Systems and methods are presented for reliable transmission of time-sensitive data. In particular, various embodiments provide for the generation of compressed sequential data, where individual instances of a sequence represent differentials from prior instances in that sequence. In order to reduce an amount of data that needs to be transmitted, instances of data (such as individual video frames) can be provided using a prior video frame as a reference, sending only data for those pixel locations where the pixel value differs from the reference frame. A reference frame can include a previously-received and successfully-decoded frame, in order to minimize the impact of dropped, incomplete, or corrupted frames. In order to further reduce data transmission requirements, a reference frame can be selected which is determined to be optimal for the current frame, such as may represent a least amount of data to be transmitted for a given frame.

First claim

What is claimed is:

1. A video streaming system, comprising:
a streaming server to transmit a stream of video data to a client device; and
an encoder to encode a sequence of video frame data to be included in the stream, wherein the encoder is to encode the sequence at least in part by:
selecting, based at least on a number of changes between a reference frame and a current progressive frame and based on a determination of a network condition, the reference frame from a set of valid frames including one or more key frames and one or more progressive frames to use for the current progressive frame in the video stream, wherein the network condition is used to determine the number of changes considered for selecting the reference frame, the reference frame being stored using a reference buffer for which video data is acknowledged to have been received by the client device; and
encoding video data for the current progressive frame including changes in pixel data relative to the selected reference frame.

2. The video streaming system of claim 1 , wherein the reference frame is further selected using at least one selection criterion that includes at least one of a data size, a recency of transmission, or a network performance state.

3. The video streaming system of claim 1 , wherein the encoder is further to receive, from the client device, acknowledgement that the current frame was properly received and decoded by the client device, and wherein data for the current frame is to be stored as an additional confirmed frame of one or more confirmed frames stored in the reference buffer.

4. The video streaming system of claim 1 , wherein the encoder is to remove, from the reference buffer, one or more confirmed frames from the reference buffer indicated to no longer be available for use as a reference frame.

5. The video streaming system of claim 1 , wherein the encoder is to apply forward error correction to the video data before the video data is transmitted to the client device.

6. The video streaming system of claim 1 , wherein the system comprises at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for rendering graphical output;
a system for performing deep learning operations;
a system implemented using an edge device;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.

7. A non-transitory computer-readable storage medium including instructions that, if executed by one or more processor, cause the one or more processors to:
select, based at least on a number of changes between a reference frame and a current progressive frame and based on a determination of a network condition, the reference frame from a set of valid frames including one or more key frames and one or more progressive frames to use for the current progressive frame in a video stream, wherein the network condition is used to determine the number of changes considered for selecting the reference frame, the reference frame corresponding to a confirmed frame, of one or more confirmed frames, stored to a reference buffer for which video data is acknowledged to have been received by a client device; and
encode video data for the current frame including changes in pixel data relative to the selected reference frame.

8. The non-transitory computer-readable storage medium of claim 7 , wherein the reference frame is further selected using at least one selection criterion includes at least one of a data size, a recency of transmission, or a network performance state.

9. The non-transitory computer-readable storage medium of claim 7 , wherein the instructions that, if executed by one or more processor, further cause the one or more processors to:
receive, from the client device, acknowledgement that the current frame has been properly received and decoded by the client device, and wherein data for the current frame is to be stored to the reference buffer as an additional confirmed frame.

10. The non-transitory computer-readable storage medium of claim 7 , wherein the instructions that, if executed by one or more processor, further cause the one or more processors to:
remove, from the reference buffer, one or more confirmed frames indicated to no longer be available for use as a reference frame.

11. A computer-implemented method, comprising:
selecting, based at least on a number of changes between a reference frame and a current progressive frame and based on a determination of a network condition, the reference frame from a set of valid frames including one or more key frames and one or more progressive frames to use for the current progressive frame in a video stream, wherein the network condition is used to determine the number of changes considered for selecting the reference frame, the reference frame corresponding to a confirmed frame, of one or more confirmed frames, stored to a reference buffer for which video data is acknowledged to have been received by a client device; and
encoding video data for the current frame including changes in pixel data relative to the selected reference frame.

12. The computer-implemented method of claim 11 , wherein the reference frame is further selected using at least one selection criterion includes at least one of a data size, or a recency of transmission.

13. The computer-implemented method of claim 11 , further comprising:
receiving, from the client device, acknowledgement that the current frame has been properly received and decoded by the client device, wherein data for the current frame is to be stored to the reference buffer as an additional confirmed frame.

14. The computer-implemented method of claim 11 , further comprising:
removing from the reference buffer, one or more confirmed frames indicated to no longer be available for use as a reference frame.

15. The computer-implemented method of claim 11 , further comprising:
applying forward error correction to the video data before the video data is transmitted to the client device.

16. The computer-implemented method of claim 11 , further comprising:
removing from the reference buffer, one or more confirmed frames including changes in pixel data that exceed a maximum differential threshold.

17. The computer-implemented method of claim 11 , wherein the reference frame is selected based, at least in part, upon a configured policy.

Assignees

• Nvidia Corp

Inventors

• Waine Benjemin Thomas
• Rose Amy
• Woodard Andrew James

Classifications

• H04N19/65

  using error resilience · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/164 Primary

  Feedback from the receiver or from the transmission channel · CPC title

• H04N19/89

  involving methods or arrangements for detection of transmission errors at the decoder · CPC title

• H04N19/61

  in combination with predictive coding · CPC title