Method and system of video coding with a multi-pass prediction mode decision pipeline

Patent metadata

Field	Value
Publication number	US-11082706-B2
Application number	US-202016800368-A
Country	US
Kind code	B2
Filing date	Feb 25, 2020
Priority date	May 5, 2016
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

Techniques related to video coding with a multi-pass prediction mode decision pipeline.

First claim

What is claimed is:

1. A computer-implemented method of video coding comprising:
obtaining a plurality of frames of a video sequence to encode and having individual frames divided into blocks of pixel data; and
generating a best prediction mode decision of individual current blocks of at least one of the frames comprising:
performing multiple prediction passes for an individual current block comprising forming a prediction mode decision comprising selecting among a plurality of candidate prediction modes at each pass of the multiple prediction passes, and
determining which one of the prediction mode decisions is a final prediction mode decision.

2. The method of claim 1 wherein the multiple prediction passes comprise a first pass, at least one middle pass, and a final pass, wherein the first, middle, and final passes each have at least one different available inter-prediction operation than the other of first, middle, and final passes.

3. The method of claim 1 wherein the multiple prediction passes are arranged to provide a first pass and at least one middle pass to individually form preliminary best prediction mode decisions, and a final pass forms the final prediction mode decision, and wherein the preliminary best mode predictions are transformable into the final prediction mode decision without performing further passes depending on the similarity of a motion vector of one pass to a motion vector of a previous pass and of the current block.

4. The method of claim 1 comprising performing at least the first pass without using data from other blocks on the same frame as the current block to determine a motion vector of the current block and of the first pass.

5. The method of claim 1 wherein the multiple prediction passes comprises a first pass, at least one middle pass, and a final pass, and wherein no current block uses block data of any block in the same frame as the current block to form a motion vector in the first pass.

6. The method of claim 5 wherein data of blocks in the same frame as the current block are limited to use in the final pass.

7. The method of claim 1 comprising providing an option to use any of the following prediction modes in the first pass to determine a predicted motion vector (PMV) of the current block: a zero motion vector, hierarchical motion estimation (HME), and the motion vector of the same block position on a previous frame relative to a current frame of the current block.

8. The method of claim 7 comprising using the PMV as a basis of at least one of integral motion estimation (IME) and fractional motion estimation (FME) in at least the first pass.

9. The method of claim 1 comprising determining a best prediction mode in a first pass among at least intra prediction, integral motion estimation (IME), and fractional motion estimation (FME) prediction modes; and
comparing prediction data of at least one prediction mode of a middle or final pass to the prediction data of a best prediction mode of the first pass.

10. The method of claim 1 wherein the operations of each pass or multiple passes are dispatched as a separate batch command.

11. A computer-implemented method of video decoding comprising:
obtaining a plurality of compressed frames of a video sequence to decode and having individual frames divided into blocks of pixel data; and
obtaining prediction data comprising a prediction mode of a current block of one of the frames, the prediction mode being selected at an encoder, at least in part, by:
performing multiple prediction passes for an individual current block comprising forming a prediction mode decision comprising selecting one among a plurality of candidate prediction modes at each pass of the multiple prediction passes, and
determining which one of the prediction mode decisions is a final prediction mode decision.

12. The method of claim 11 wherein the multiple prediction passes comprise a first pass, at least one middle pass, and a final pass, wherein the first, middle, and final passes each have at least one different available inter-prediction operation than the other of first, middle, and final passes.

13. The method of claim 11 comprising, in at least one middle pass of the multiple prediction passes, using a predicted motion vector (PMV) from a previous pass to compare to a PMV of the current middle pass of the multiple prediction passes to determine a prediction mode decision in the current middle pass.

14. The method of claim 11 comprising omitting a final pass that uses data of other blocks in the same frame as a current block when it is determined that a result of a prediction mode decision of the first pass or a middle pass is to be the final prediction mode decision depending on a comparison of predicted motion vectors (PMVs) of two different passes.

15. The method of claim 11 comprising, in a middle pass, determining a best prediction mode based, at least in part, on whether a prediction motion vector (PMV) of a previous pass is the same or similar to a current predicted motion vector (PMV) of the current block, wherein the current PMV is based on, at least in part, blocks nearby to a position on a prior frame that corresponds to the same position of the current block on the current frame.

16. A computer-implemented system comprising:
at least one display;
at least one memory to store frames of a video sequence wherein individual frames have image data with pixels divided into blocks;
at least one processor communicatively coupled to the at least one of the memory and display, and arranged to operate by:
generating a best prediction mode decision of individual current blocks of one of the frames comprising:
performing multiple prediction passes for an individual current block comprising forming a prediction mode decision comprising selecting one among a plurality of candidate prediction modes at each pass of the multiple prediction passes, and

determining which one of the prediction mode decisions is a final prediction mode decision.

17. The system of claim 16 wherein the multiple prediction passes comprise a first pass, at least one middle pass, and a final pass, wherein the first, middle, and final passes each have at least one different available inter-prediction operation than the other passes.

18. The system of claim 16 wherein the at least one processor is to operate by determining a best prediction mode based, at least in part, on whether a distortion of a current best prediction mode is less than or equal to the distortion of a best prediction mode on a previous pass.

19. The system of claim 18 wherein a determination that a best prediction mode distortion of a current pass is less than or equal to the distortion of the best prediction mode of a prior pass that indicates that intra coding is the final best prediction mode of the current block avoids performance of any more integer motion estimation to determine a motion vector of the current block.

20. At least one non-transitory computer-readable medium having stored thereon instructions that when executed cause a computing device to operate by:
obtaining a plurality of frames of a video sequence to encode wherein individual frames are divided into blocks of pixel data; and
generating a best prediction mode decision of individual current blocks of one of the frames comprising:
performing multiple prediction passes for an individual current block comprising forming a prediction mode decision comprising selecting one among a plurality of candidate prediction modes at each pass

Assignees

• Intel Corp

Inventors

• Tanner Jason
• Patel Jay M

Classifications

• H04N19/105

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/139

  Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability · CPC title

• H04N19/567

  Motion estimation based on rate distortion criteria · CPC title

• H04N19/53

  Multi-resolution motion estimation; Hierarchical motion estimation · CPC title

• H04N19/176 Primary

  the region being a block, e.g. a macroblock · CPC title