Coding tree-based adaptive quantization

Patent metadata

Field	Value
Publication number	US-2025310572-A1
Application number	US-202418637165-A
Country	US
Kind code	A1
Filing date	Apr 16, 2024
Priority date	Mar 27, 2024
Publication date	Oct 2, 2025
Grant date	—

Abstract

Systems and methods herein are for a video encoder to be associated with a temporal filter and a coding tree and that can perform a main pass for video encoding using individual video blocks towards prediction of at least one frame associated with the media stream, where the coding tree is associated with a lookahead pass, and where the temporal filter can enable denoising within the lookahead pass to reduce an effect of noise in one or more of motion estimation or mode selection of the video encoding.

First claim

What is claimed is:

1 . A video encoder to be associated with a temporal filter and a coding tree and to perform a main pass for video encoding using individual video blocks towards prediction of at least one frame associated with a media stream, wherein the coding tree is associated with a lookahead pass, and wherein the temporal filter is to enable denoising within the lookahead pass to reduce an effect of noise on one or more of a motion estimation or mode selection of the video encoding.

2 . The video encoder of claim 1 , wherein the denoising is a reference frame denoising that is associated with a reference frame from a group of frames (GoF) that includes the at least one frame.

3 . The video encoder of claim 1 , wherein the temporal filter and the coding tree are performed by separate features of the video encoder and wherein an output of the coding tree enables the mode selection using a rate distortion optimization (RDO) feature of the video encoder.

4 . The video encoder of claim 1 , wherein the temporal filter and the coding tree are part of a driver that is adapted to receive at least part of the media stream and that is adapted to provide a lookahead output to the video encoder, the video encoder to provide an output bitstream that is based in part on the lookahead output and that is a compression of the media stream, the output bitstream comprising a bit rate savings over a threshold that is associated with a reference compression comprising only the main pass performed on the media stream.

5 . The video encoder of claim 1 , wherein an output of the coding tree is to enable the mode selection to provide a decision for compression of the media stream based in part on the reduction to the effect of the noise in the video encoding associated with the main pass.

6 . The video encoder of claim 1 , wherein the video encoder is adapted for H.264, HEVC, and AV1 encoding, wherein the coding tree is a macroblock (MB) tree or a coding unit (CU) tree, superblock (SB) tree and wherein the individual video blocks are individual macroblocks (MBs) or individual coding units (CUs).

7 . A system comprising:
one or more processing units to be associated with a temporal filter and a coding tree and to perform a main pass for video encoding using individual video blocks towards prediction of at least one frame associated with a media stream, wherein the coding tree is associated with a lookahead pass, and wherein the temporal filter is to enable denoising within the lookahead pass to reduce an effect of noise on one or more of a motion estimation or mode selection of the video encoding.

8 . The system of claim 7 , wherein the denoising is a reference frame denoising that is associated with a reference frame from a group of frames (GoF) that includes the at least one frame.

9 . The system of claim 7 , wherein the temporal filter and the coding tree are performed by separate features of the video encoder and wherein an output of the coding tree enables the mode selection using a rate distortion optimization (RDO) feature of the video encoder.

10 . The system of claim 7 , wherein the temporal filter uses an inference of the noise between frames from a group of frames (GoF) and provides the denoising based in part on the inference.

11 . The system of claim 7 , wherein the temporal filter and the coding tree are part of a driver that is adapted to receive at least part of the media stream and that is adapted to provide a lookahead output to the video encoder, the video encoder to provide an output bitstream that is based in part on the lookahead output and that is a compression of the media stream, the output bitstream comprising a bit rate savings over a threshold that is associated with a reference compression comprising only the main pass performed on the media stream.

12 . The system of claim 11 , wherein an output of the coding tree is to enable the mode selection to provide a decision for compression of the media stream based in part on the reduction to the effect of the noise in the video encoding associated with the main pass.

13 . A system comprising:
one or more processing units to perform a compression of a media stream based in part on a main pass for video encoding using individual video blocks towards prediction of at least one frame associated with the media stream and based in part on a lookahead pass that is associated with a coding tree and a temporal filter, the temporal filter to enable denoising within the lookahead pass to reduce an effect of noise on one or more of a motion estimation or mode selection of the video encoding.

14 . The system of claim 13 , wherein the denoising is a reference frame denoising that is associated with a reference frame from a group of frames that includes the at least one frame.

15 . The system of claim 13 , wherein the temporal filter and the coding tree are performed by separate features of the video encoder and wherein an output of the coding tree enables the mode selection using a rate distortion optimization (RDO) feature of the video encoder.

16 . A method for a video encoder, the method comprising:
enabling a temporal filter and a coding tree for the video encoder; performing a main pass of the coding tree for video encoding, using individual video blocks, towards prediction of at least one frame associated with the media stream; enabling a lookahead pass that is associated with the coding tree; and performing denoising within the lookahead pass using the temporal filter, the denoising to reduce an effect of noise on one or more of a motion estimation or mode selection of the video encoding.

17 . The method of claim 16 , further comprising:
performing the temporal filter and the coding tree by separate features of the video encoder; and using an output of the coding tree for the mode selection by a rate distortion optimization (RDO) feature of the video encoder.

18 . The method of claim 16 , further comprising:
performing the temporal filter and the coding tree using a driver that is adapted to receive at least part of the media stream; enabling a lookahead output to the video encoder from the driver; and providing an output bitstream from the video encoder based in part on the lookahead output, the output bitstream to comprise a compression of the media stream and to comprise a bit rate savings over a threshold that is associated with a reference compression comprising only the main pass performed on the media stream.

19 . The method of claim 16 , wherein the denoising is a reference frame denoising that is associated with a reference frame from a group of frames (GoF) that includes the at least one frame.

20 . The method of claim 16 , wherein an output of the coding tree is to enable the mode selection to provide a decision for compression of the media stream based in part on the reduction to the effect of the noise in the video encoding associated with the main pass.

Assignees

• Nvidia Corp

Inventors

• Porat Dror
• Levi Dotan David
• Martin Limor
• Parashar Vipul
• Gupta Yogender
• Mishra Sampurnananda
• Chen Jianjun

Classifications

• H04N19/117

  Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) · CPC title

• H04N19/124

  Quantisation · CPC title

• H04N19/80

  Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence) · CPC title

• H04N19/96

  Tree coding, e.g. quad-tree coding · CPC title

• H04N19/103

  Selection of coding mode or of prediction mode · CPC title