Fine grain lookahead enhancement for video coding

Patent metadata

Field	Value
Publication number	US-12348734-B2
Application number	US-202117347243-A
Country	US
Kind code	B2
Filing date	Jun 14, 2021
Priority date	Jun 14, 2021
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

Techniques related to video coding using look ahead analysis and block based back propagation for block level quantization parameters are discussed. Such techniques include adaptively selecting frames subsequent to a target frame in a display order for use in the back propagation, propagating values to blocks of the target frame that are indicative of the importance of the blocks to the encode of the subsequent frames, and encoding the target frame using block level quantization parameters based on the propagated values.

First claim

What is claimed is:

1. A device comprising:
a memory to store at least a portion of input video, the input video comprising a target frame and subsequent frames in a display order; and
processor circuitry coupled to the memory, the processor circuitry to perform operations including:
performing lookahead analysis for the input video;
selecting one or more of the subsequent frames for use in a propagation of block based propagation values for adjustment of block level quantization parameters (QPs) of the target frame;
propagating the block based propagation values from the selected one or more of the subsequent frames to the target frame, wherein the block based propagation values are based on spatial and temporal distortion values from the lookahead analysis of the input video, and the block based propagation values are propagated using motion estimation based on the target frame and the selected one or more of the subsequent frames;
wherein propagating the block based propagation values comprises:
determining that a first block based propagation value from a first block of a first frame of the subsequent frames is propagated to a second block of a second frame of the subsequent frames; and
scaling, in response to a first spatial distortion of the first block of the first frame exceeding a second spatial distortion of the second block of the second frame, a block based propagation value for the second block relative to a ratio of the second spatial distortion to the first spatial distortion;

selecting the block level QPs of the target frame based on the block based propagation values corresponding to blocks of the target frame; and
encoding the target frame using the block level QPs.

2. The device of claim 1 , wherein selecting the one or more of the subsequent frames comprises:
determine a scene change frame in the subsequent frames; and
selecting the one or more frames only from frames prior to the scene change frame in the display order.

3. The device of claim 1 , wherein:
selecting the one or more of the subsequent frames comprises detecting a complexity change greater than a threshold between a third frame and a fourth frame of the subsequent frames; and
propagating the block based propagation values comprises propagating the block based propagation values from only one or more frames prior to and including the third frame and propagating further block based propagation values from only one or more frames subsequent to and including the fourth frame.

4. The device of claim 1 , wherein;
selecting the one or more of the subsequent frames comprises selecting the one or more frames at an interval that skips one or more intervening frames; and
propagating the block based propagation values absent use of the intervening frames.

5. The device of claim 1 , wherein:
the subsequent frames comprise a first temporal complexity measure;
the selected one or more of the subsequent frames comprise a third frame at a first maximum temporal distance from the target frame; and
the operations further include:
selecting, for a second target frame, one or more of a plurality of second subsequent frames for use in a second propagation for adjustment of second block level QPs of the second target frame, the second subsequent frames comprising a second temporal complexity measure and the selected one or more second subsequent frames comprising a third frame at a second maximum temporal distance from the second target frame, wherein the second maximum temporal distance is greater than the first maximum temporal distance in response to the second temporal complexity measure being greater than the first temporal complexity measure.

6. The device of claim 1 , wherein:
propagating the block based propagation values comprises determining that the first block based propagation value from the first block of the first frame is propagated to a third block of the second frame; and
a second block based propagation value for the third block is not scaled in response to the first spatial distortion of the first block not exceeding a third spatial distortion of the third block.

7. The device of claim 1 , wherein:
the lookahead analysis uses downsampled input video and the target frame has a first distortion value based on the lookahead analysis; and
the operations further include:
generating a second distortion value for the target frame based on full resolution input video; and
increasing, in response to a ratio of the second distortion value to the first distortion value exceeding a threshold, one or more block level QPs prior to said encoding.

8. The device of claim 1 , wherein selecting the block level QPs comprises:
receiving first block level QPs for blocks of the target frame, the first block level QPs comprising a first number of unique block level QPs, the first number exceeding a threshold; and
repeatedly altering block level QP values of a set of the first block level QPs corresponding to a lowest number of unique block level QPs until an updated a number of unique block level QPs does not exceed the threshold.

9. The device of claim 8 , wherein repeatedly altering the block level QP values of the set of the first block level QPs comprises moving the block level QP values to a lower QP value.

10. The device of claim 1 , wherein:
the lookahead analysis comprises a coding order matching the display order;
the target frame and the subsequent frames comprise a group of picture (GOP) structure having a further coding order other than the display order; and
propagating the block based propagation values comprises:
determining a number of forward and backward reference frames in the GOP structure for a third frame of the subsequent frames, at least one of the forward and backward reference frames skipping one or more frames in the display order; and
propagating second block based propagation values, using the lookahead analysis, to the third frame from the forward and backward reference frames in the display order and using the number of the forward and backward reference frames based on the GOP structure.

11. The device of claim 10 , wherein propagating the block based propagation values comprises:
dividing a first block based propagation value of a block of a first forward frame into two or more values for corresponding blocks of the third frame; and
merging second block based propagation values of blocks of a first backward frame into a single value for a corresponding block of the third frame.

12. A method for video coding comprising:
performing lookahead analysis for input video, wherein the input video comprises a target frame and subsequent frames in a display order, the lookahead analysis uses downsampled input video, and the target frame has a first distortion value based on the lookahead analysis;
generating a second distortion value for the target frame based on full resolution input video;
selecting one or more of the subsequent frames for use in a propagation of block based propagation values for adjustment of block level quantization parameters (QPs) of the target frame;
propagating the block based propagation values from the selected one or more of the subsequent frames to the target frame, wherein the block based propagation values are based on spatial and temporal distortion values from the lookahead analysis of the input video, and the block based propagation values are propagated using motion estimation based on the target frame and the selected one or more of the subsequent frames;
selecting the block level QPs of the target frame based on the block based propagation values;
increasing, in response to a ratio of the second di

Assignees

• Intel Corp

Inventors

• Tanner Jason
• Holland James

Classifications

• H04N19/126 Primary

  Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers · CPC title

• H04N19/177

  the unit being a group of pictures [GOP] · CPC title

• H04N19/196

  being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513) · CPC title

• H04N19/577

  Motion compensation with bidirectional frame interpolation, i.e. using B-pictures · CPC title

• H04N19/139

  Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability · CPC title