Complexity allocation for video and image coding applications

Patent metadata

Field	Value
Publication number	US-9479786-B2
Application number	US-200913119492-A
Country	US
Kind code	B2
Filing date	Sep 24, 2009
Priority date	Sep 26, 2008
Publication date	Oct 25, 2016
Grant date	Oct 25, 2016

Abstract

Coding method complexity types are assigned to video for combinations of coding decision levels associated with a sequence of pictures, a group of pictures, a scene, a region of the scene or picture, a block or macroblock. Multiple coding units are encoded as a function of complexity and determined from the coding decision levels. A first coding decision level is associated with determining a first complexity and a first coding method type for the sequence of pictures. A second coding decision level is associated with determining a second complexity and a second coding method type for the pictures or scene. The picture or sequence includes the coding units. A third coding decision level is associated with determining a third complexity and a third coding method type for the coding units.

First claim

What is claimed is:

1. A method for coding video with a complexity-scalable coding mode decision scheme for a video coding system, the method comprising:
identifying at least a spatial area associated with the video;
determining one or more characteristics of at least the spatial area, wherein the one or more characteristics include statistical characteristics;
assigning a complexity level to at least the spatial area using the one or more determined characteristics, wherein the complexity level at least relates to computational resources for the video coding system, wherein the assigning comprises assigning one or more coding method types for the complexity level for any one of or any combination of coding decision levels, wherein the coding decision levels comprise a first coding decision level, a second coding decision level, and a third coding decision level, wherein the coding decision levels are hierarchically arranged, and wherein the video coding system is configured for at least a random assignment coding method; and
coding a plurality of coding units as a function of the complexity level and the one or more coding method types determined from one or more of the coding decision levels, wherein the coding units comprise a macroblock of pixels or a sequence of macroblocks for the spatial area;
wherein:
the coding decision levels are associated with a sequence of pictures, a group of pictures (GOP), at least one of the pictures, a scene of the sequence of pictures, a region of the scene or the at least one picture, a block, or the macroblock,
the first coding decision level is associated with determining a first complexity level and a first coding method type for the sequence of pictures, wherein the first coding decision level comprises a sequence coding level,
the second coding decision level is associated with determining a second complexity level and a second coding method type for the at least one of the pictures or the scene of the sequence of pictures, wherein the at least one picture or the sequence of pictures comprises the plurality of coding units, and wherein the second coding decision level comprises a picture coding level or a scene coding level, and
the third coding decision level is associated with determining a third complexity level and a third coding method type for the plurality of coding units, wherein the third coding decision level comprises a coding unit coding level,

wherein the random assignment coding method includes:
randomly encoding the plurality of coding units, and
assigning randomly-generated numbers to the plurality of coding units.

2. The method of claim 1 , further comprising:
in the first coding decision level, allocating a plurality of coding parameters, wherein the plurality of coding parameters comprises a pattern decision parameter, a scene parameter, and a map parameter, wherein the allocating comprises:
allocating the pattern decision parameter for complexity allocation at the sequence coding level;
allocating the scene parameter to indicate a use of scene analysis for any of the determinations; and
allocating the map parameter to indicate a pattern map.

3. The method of claim 2 , further comprising:
in the first coding decision level,
allocating any of the parameters in response to a user input or allocating any of the parameters using one or more statistics, wherein the sequence of macroblocks correspond to a coding slice, a rate control unit, or a pattern corresponding to an object region;
selecting at least one of the coding method types based on the one or more allocated parameters, wherein at least one of the parameters in the plurality of parameters indicates whether every picture in the sequence is to be coded with a same coding method type,

wherein the video coding system comprises an encoder or a decoder;
wherein any of the coding method types comprises one or more statistical characteristics associated with the encoder or the decoder.

4. The method of claim 3 , wherein, in the first coding decision level, an adaptive assignment coding method utilizes one or more coding method types and one or more coding patterns for one or more pictures in the sequence or a scene of the sequence,
wherein the adaptive assignment coding method is dependent on a characteristic of an image, a slice type, a prediction mode, a brightness level, a variance, an object analysis, motion, a picture distance from a last scene change, a scene analysis, or one or more characteristics of the encoder.

5. The method of claim 4 further comprising:
in the first coding decision level, determining if scene analysis is enabled or disabled, wherein if scene analysis is disabled, the second coding decision level utilizes picture-level complexity allocation, and wherein if scene analysis is enabled, the second coding decision level uses scene-level complexity allocation, wherein for the picture-level complexity allocation a plurality of pictures are encoded with a predetermined coding method.

6. The method of claim 5 , wherein:
the scene-level complexity allocation comprises the picture-level complexity allocation, or
the scene-level complexity allocation comprises classifying one or more transitions in the sequence, wherein the one or more transitions comprises scene changes, fade-ins, fade-outs, or cross-fading, and wherein the scene-level complexity allocation further comprises assigning one or more coding method types to different scenes.

7. The method of claim 6 , wherein, in the second coding decision level, the method comprises:
determining whether the one or more parameters from the first coding decision level are to be used in the second coding decision level;
if one or more parameters from the first coding decision level are determined to be used in the second coding decision level, copying one or more parameters from the first coding decision level in the second coding decision level to provide access to the one or more copied parameters in the second coding decision level; and
selecting at least one of the coding method types for the at least one of the pictures or the scene of the sequence of pictures based on the one or more allocated parameters.

8. The method of claim 7 , wherein the selection of the random assignment coding method for the at least one of the pictures or the scene of the sequence of pictures comprises randomly encoding the plurality of coding units in the picture or scene;
assigning the randomly-generated numbers to the plurality of coding units using rate distortion optimization (RDO) complexity levels corresponding to the randomly-generated numbers; and
classifying the complexity levels using an estimate or a value of a system metric, wherein the system metric comprises an evaluation of the complexity levels with an amount of memory, a number of required operations, a required amount of time, or a required number of cycles, wherein the randomly-generated numbers are based on a probability or a distribution.

9. The method of claim 8 , wherein the selection of a fixed assignment coding method for the at least one of the pictures or the scene of the sequence of pictures comprises encoding the plurality of coding units in the picture or scene with the fixed assignment coding method,
wherein the fixed assignment coding method comprises using a fixed coding pattern for the picture or scene, and
wherein the fixed coding pattern comprises a column-wise pattern, a checker-board pattern, a scan-order pattern, a spiral pattern, a zig-zag pattern, an irregular pattern, a pattern with a plurality of complexity levels, a pattern with random complexity levels, a user-defined pattern, a system-defined pattern, or any combination of the patterns.

Assignees

• Lu Limin
• Tourapis Alexandros
• Dolby Laboratories Licensing Corp

Inventors

• Lu Limin
• Tourapis Alexandros

Classifications

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/127

  Prioritisation of hardware or computational resources · CPC title

• H04N19/19 Primary

  using optimisation based on Lagrange multipliers · CPC title

• H04N19/61

  in combination with predictive coding · CPC title