Multiple color channel multiple regression predictor

Patent metadata

Field	Value
Publication number	US-10237552-B2
Application number	US-201815988937-A
Country	US
Kind code	B2
Filing date	May 24, 2018
Priority date	Apr 14, 2011
Publication date	Mar 19, 2019
Grant date	Mar 19, 2019

Abstract

Inter-color image prediction is based on multi-channel multiple regression (MMR) models. Image prediction is applied to the efficient coding of images and video signals of high dynamic range. MMR models may include first order parameters, second order parameters, and cross-pixel parameters. MMR models using extension parameters incorporating neighbor pixel relations are also presented. Using minimum means-square error criteria, closed form solutions for the prediction parameters are presented for a variety of MMR models.

First claim

What is claimed is:

1. In an encoder comprising a processor, a method to approximate an image having a first dynamic range in terms of an image having a second dynamic range, the method comprising;
receiving a first image and a second image, wherein the second image has a different dynamic range than the first image;
determining values of prediction parameters for a multi-channel, multiple-regression (MMR) prediction model, wherein pixel values of at least one color component in a predicted image approximating the first image are computed based on pixel values of at least two color components in the second image;
outputting the determined values of the prediction parameters, wherein for generating a predicted pixel value (OD for a luma and/or chroma component the MMR model comprises: a constant value (n), at least a linear combination of pixel values of two different color components in the second image, and at least one product term of pixel values of two different color components in the second image.

2. The method of claim 1 , wherein the MMR model computes the predicted pixel value (OD according to the formula:
{circumflex over (v)} i =n+Σ k=1 3 m k (1) s ik +mc 1 (1) s i1 s i3 +mc 3 (1) s i2 s i3 +mc 4 (1) s i1 s i2 s i3 ,
where s ik , k=1, 2, and 3, denote the three color components of the i-th pixel of the second image, and m k (1) (for k=1, 2, and 3), n, and mc j (1) ((j=1, 2, 3, 4) denote the prediction parameters of the MMR model.

3. The method of claim 1 , wherein the dynamic range of the first image is higher than the dynamic range of the second image.

4. The method of claim 3 , wherein the first image is a high dynamic range image and the second image is a standard dynamic range image.

5. The method of claim 1 , wherein the MMR model further comprises at least a square of the pixel value of one of the color components in the second image and at least a square of a product of pixel values of two different color components in the second image to form a second MMR model with cross products.

6. The method of claim 5 , wherein the MMR model computes the predicted pixel value ({circumflex over (v)} i ) according to the formula:
{circumflex over (v)} i =n+Σ k=1 3 m k (1) s ik +mc 1 (1) s i1 s i2 +mc 2 (1) s i1 s i3 +mc 3 (1) s i2 s i3 +mc 4 (1) s i1 s i2 s i3 +Σ k=1 3 m k (2) s ik 2 +mc 1 (2) ( s i1 s i2 ) 2 +mc 2 (2) ( s i1 s i3 ) 2 +mc 3 (2) ( s i2 s i3 ) 2 +mc 4 (2) ( s i1 s i2 s i3 ) 2 ,

where s ik , k=1, 2, and 3, denote the three color components of the i-th pixel of the second image, and m k (1) , m k (2) (for k=1, 2, and 3), n, mc j (1) , mc j (2) (j=1, 2, 3, 4) denote the prediction parameters of the MMR model.

7. The method of claim 5 , wherein the MMR model further comprises at least a cube of the pixel value of one of the color components in the second image and at least a cube of a product of pixel values of two different color components in the second image to form a third order MMR model with cross products.

8. The method claim 7 , wherein the MMR model computes the predicted pixel value (OD according to the formula:
{circumflex over (v)} i =n+Σ k=1 3 m k (1) s ik +mc 1 (1) s i1 s i2 +mc 2 (1) s i1 s i3 +mc 4 (1) s i1 s i2 s i3 +Σ k=1 3 m k (2) s ik 2 +mc 1 (2) ( s i1 s i2 ) 2 +mc 2 (2) ( s i1 s i3 ) 2 +mc 3 (2) ( s i2 s i3 ) 2 +mc 4 (2) ( s i1 s i2 s i3 2 +Σ k=1 3 m k (3) s ik 3 +mc 1 (3) ( s i1 s i2 ) 3 +mc 2 3 (s i1 s i3 ) 3 +mc 3 (3) ( s i2 s i3 ) 3 +mc 4 (3) ( s i1 s i2 s i3 ) 3 ,

where s ik , k=1, 2, and 3, denote the three color components of the i-th pixel of the second image, and m k (1) , m k (2) , m k (3) (for k=1, 2, and 3), n, mc j (1) , mc j (2) , mc j (3) (j=1, 2, 3, 4) denote the prediction parameters of the MMR model.

9. The method of claim 1 wherein determining the values of the prediction parameters of the MMR prediction model further comprises applying numerical methods that minimize the mean square error between the first image and the predicted image.

10. The method of claim 1 , further comprising:
compressing with a processor the second image to generate a compressed second image; and
generating a bitstream comprising the compressed second image and the prediction parameters.

11. The method of claim 10 further comprising:
computing an output image approximating the first image based on the second image and the determined values of the prediction parameters of the MMR prediction — model; and
computing a coded residual image based on the output image and the first image; and
generating the bitstream based on the compressed second image, the coded residual image, and the prediction parameters.

12. In an encoding or decoding video system, an image prediction method with a processor, the method comprising:
receiving a first image;
receiving prediction parameters for a multi-channel multiple-regression (MMR) prediction model, wherein the MMR model is adapted to predict a second image in terms of the first image; and
applying the first image and the prediction parameters to the MMR prediction model to generate an output image approximating the second image, wherein pixel values of at least one color component of the output image are computed based on pixel values of at least two color components in the first image, wherein for generating a predicted output pixel value ({circumflex over (v)} i ) for a luma and/or chroma component the MMR model comprises: a constant value (n) , at least a linear combination of pixel values of two different color components in the first image, at least one product term of pixel values of two different color components in the first image, at least a square of the pixel value of one of the color components in the first image, and at least a square of a product of pixel values of two different color components in the first image.

13. The method of claim 12 , wherein the MMR model computes the predicted output pixel value ({circumflex over (v)} i ) according to the formula:
{circumflex over (v)} i =n+Σ k=1 3 m k (1) s ik +mc 1 (1) s i1 s i2 +mc 2 (1) s i1 s i3 +mc 3 (1) s i2 s i3 +mc 4 (1) s i1 s i2 s i3 +Σ k=1 3 m k (2) s ik 2 +mc 1 (2) ( s i1 s i2 ) 2 +mc 2 (2) ( s i1 s i3 ) 2 +mc 3 (2) ( s i2 s i3 ) 2 +mc 4 (2) ( s i1 s i2 s i3 ) 2 ,

where s ik , k=1, 2, and 3, denote the three color components of the i-th pixel of the first image, and m k (1) , m k (2) (for k=1, 2, and 3), n, mc j (1) , mc j (2) (j=1, 2, 3, 4) denote the prediction parameters of the MMR model.

14. The method of claim 12 , wherein the MMR model further comprises at least a cube of the pixel value of one of the color components in the first image and at least a cube of a product of pixel values of two different color components in the first image to form a third order MMR model with cross products.

15. The method claim 14 , wherein the MMR model computes the predicted output pixel value ({circumflex over (v)} i ) according to the formula:
{circumflex over (v)} i =n+Σ k=1 3 m k (1) s ik +mc 1 (1) s i1 s i2 +mc 2 (1) s i1 s

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Su Guan-Ming
• Qu Sheng
• Koepfer Hubert
• Yuan Yufei
• Hulyalkar Samir

Classifications

• H04N19/105 Primary

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/16

  for a given display mode, e.g. for interlaced or progressive display mode · CPC title

• H04N19/192

  the adaptation method, adaptation tool or adaptation type being iterative or recursive · CPC title

• H04N19/147

  according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) · CPC title

• G06F17/18

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title