Multiple-pass rate control for video coding applications

Patent metadata

Field	Value
Publication number	US-9467693-B2
Application number	US-201113520883-A
Country	US
Kind code	B2
Filing date	Jan 3, 2011
Priority date	Jan 6, 2010
Publication date	Oct 11, 2016
Grant date	Oct 11, 2016

Abstract

Methods and systems for coding applications are described, where multiple coding passes are performed for purposes of rate control. Prediction structures are generated and used by the multiple coding passes to update coding statistics generated prior to the coding passes. The updated coding statistics are used for a subsequent coding operation.

First claim

The invention claimed is:

1. A method for coding video information in multiple coding passes to control rate of the video information, comprising:
optimizing one or more low computational complexity coding passes for one or more of: coding efficiency, transmission efficiency, and scalability of the one or more low computational complexity coding passes;
applying the optimized one or more low computational complexity coding passes to the video information;
collecting initial statistics on the coded video information based on the one or more low computational complexity coding passes;
optimizing intermediate coding passes for bit usage of the intermediate coding passes, the intermediate coding passes comprising prediction structures;
applying the optimized intermediate coding passes to the coded video information;
analyzing bit usage of the applied optimized intermediate coding passes, and coding performance of the prediction structures, the prediction structures comprising a sequence of parameters, the parameters describing a type and coding order for each frame in the prediction structures, the type comprising at least one of: an intra-frame, a uni-predictive inter-frame, a bi-predictive inter-frame, and a disposable frame;
collecting updated statistics on the coded video information based on the analyzed bit usage and analyzed coding performance;
detecting coding inaccuracies in the coded video information resulting from the optimized one or more low computational complexity coding passes;
setting a full complexity coding pass at a computational complexity higher than the optimized intermediate coding passes;
optimizing the full complexity coding pass to remove the detected coding inaccuracies and for one or more of: coding efficiency, transmission efficiency, and scalability of the full complexity coding pass, based on the initial statistics and the updated statistics; and
applying the optimized full complexity coding pass to the coded video information.

2. The method as recited in claim 1 , further comprising generating the prediction structures based on the video information and minimizing a length of the prediction structures.

3. The method as recited in claim 2 , wherein generating the prediction structures comprises:
generating prediction structure atoms;
determining positions, inside the video information, where the prediction structure atoms should be inserted to optimize coding performance of the prediction structures;
optimizing coding performance of the prediction structure atoms; and
generating new prediction structures using different prediction structure atoms;
wherein each prediction structure atom comprises a smallest possible prediction unit.

4. The method as recited in claim 3 , wherein generating the prediction structures further comprises:
generating, in addition to the initial prediction structure atoms, additional prediction structure atoms drawn from prediction structures to be used for the further coding pass of the video information.

5. The method as recited in claim 3 , wherein the prediction structure atoms are extended to larger prediction structures.

6. The method as recited in claim 1 , wherein the one or more low computational complexity coding passes are asymmetric in terms of coding complexity and/or coded frames.

7. The method as recited in claim 1 , wherein the one or more low computational complexity coding passes have a computational complexity lower than the intermediate coding passes.

8. The method as recited in claim 1 , wherein each coding pass comprises coding a plurality of frames from the video information.

9. The method as recited in claim 1 , wherein the intermediate coding passes comprises temporal and spatial subsampling.

10. The method as recited in claim 9 , wherein the intermediate coding passes subsample frames in an even spaced manner or based on pre-analysis information.

11. The method as recited in claim 1 , wherein the intermediate coding passes are content-based through selection of the most representative frames of the video information.

12. The method as recited in claim 1 , wherein the intermediate coding passes are content-based through selection of those frames whose absence would cause the largest drop in rate-distortion performance.

13. The method as recited in claim 1 , wherein the intermediate coding passes comprises a plurality of coding passes, wherein each coding pass codes different groups of frames of the video information.

14. The method as recited in claim 13 , wherein the video information comprises multiple sub-sampled groups of frames and wherein:
statistics derived from one pass are adapted to be used in subsequent intermediate coding passes.

15. The method as recited in claim 13 , wherein the video information contains multiple sub-sampled groups of frames and wherein:
statistics from one group are adapted to be used in a different group.

16. The method as recited in claim 15 , wherein the different group comprises a nearby group.

17. The method as recited in claim 13 , wherein each coding pass comprises:
selection of a frame subset of the video information; and
selection of a prediction structure for the video information.

18. The method as recited in claim 1 , wherein statistics derived from one pass are adapted to be used in subsequent intermediate coding passes.

19. The method as recited in claim 1 , further comprising, prior to the intermediate coding passes:
examining the video information to detect one or more of:
intra frames;
instantaneous decoder refresh (IDR) frames;
scene changes; or
fade transitions.

20. The method as recited in claim 19 , wherein the positions of one or more of the detected intra frames, IDR frames, scene changes or fade transitions are used to build prediction structures using prediction structure atoms.

21. The method as recited in claim 1 , further comprising:
storing the initial statistics and the updated statistics.

22. The method as recited in claim 1 , wherein the applying the optimized full complexity coding pass occurred over a duration of time, the applying the optimized full complexity coding pass comprising:
generating causal coding statistics throughout the duration of time of the applying the optimized full complexity coding pass; and
applying the optimized full complexity coding pass based on the causal coding statistics.

23. A video information coding system comprising:
one or more low computational complexity coders configured to collect initial statistics on the coded video information by performing low computational complexity coding passes;
one or more intermediate coders, each intermediate coder configured to perform intermediate coding passes of the video information;
a rate control analysis module configured to:
optimize the intermediate coding passes for bit usage of the intermediate coding passes, the intermediate coding passes comprising prediction structures;
analyze bit usage of applied optimized intermediate coding passes, and coding performance of the prediction structures, the prediction structures comprising a sequence of parameters, the parameters describing a type and coding order for each frame in the prediction structures, the type comprising at least one of: an intra-frame, a uni-predictive inter-frame, a bi-predictive inter-frame, and a disposable frame;
collect updated statistics on coded video information based on the analyzed bit usage and analyzed coding pe

Assignees

• Leontaris Athanasios
• Tourapis Alexandros
• Dolby Laboratories Licensing Corp

Inventors

• Leontaris Athanasios
• Tourapis Alexandros

Classifications

• H04N19/124

  Quantisation · CPC title

• H04N19/14

  Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/192

  the adaptation method, adaptation tool or adaptation type being iterative or recursive · CPC title

• H04N19/132

  Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking · CPC title