Multiple color channel multiple regression predictor

Patent metadata

Field	Value
Publication number	US-9386313-B2
Application number	US-201414283845-A
Country	US
Kind code	B2
Filing date	May 21, 2014
Priority date	Apr 14, 2011
Publication date	Jul 5, 2016
Grant date	Jul 5, 2016

Abstract

Inter-color image prediction is based on multi-channel multiple regression (MMR) models. Image prediction is applied to the efficient coding of images and video signals of high dynamic range. MMR models may include first order parameters, second order parameters, and crosspixel parameters. MMR models using extension parameters incorporating neighbor pixel relations are also presented. Using minimum means-square error criteria, closed form solutions for the prediction parameters are presented for a variety of MMR models.

First claim

The invention claimed is:

1. A method to approximate using a processor an image having a first dynamic range in terms of an image having a second dynamic range, the method comprising;
receiving a first image and a second image, wherein the second image has a different dynamic range than the first image;
selecting a multi-channel, multiple-regression (MMR) prediction model from one or more MMR models;
determining values of prediction parameters of the selected MMR model;
computing an output image approximating the first image based on the second image and the determined values of the prediction parameters of the selected MMR prediction model, wherein pixel values of at least one color component in the output image are computed based on pixel values of at least two color components in the second image; and
outputting the determined values of the prediction parameters and the computed output image
wherein selecting the MMR prediction model from the one or more MMR prediction models further comprises an iterative selection process comprising:
(a) selecting and applying an initial MMR prediction model;
(b) computing a residual error between the first image and the output image;
(c) selecting the initial MMR model if the residual error is smaller than an error threshold and no further MMR prediction model is selectable; otherwise,

selecting a new MMR prediction model from the variety of MMR prediction models, the new MMR prediction model being different from the previously selected MMR prediction model; and returning to step (b).

2. The method of claim 1 wherein the first image comprises a Visual Dynamic Range (VDR) image and the second image comprises a Standard Dynamic Range (SDR) image.

3. The method of claim 1 wherein the selected MMR prediction model is at least one of: a first order MMR model, a second order MMR model, a third order MMR model, a first order with cross-product MMR model, a second order with cross-product MMR model, or a third order with cross-product MMR model.

4. The method of claim 3 , wherein any of the MMR models for predicting a pixel of the output image, further comprises prediction parameters related to neighbor pixels of a corresponding pixel in the second image.

5. The method of claim 4 wherein the neighbor pixels comprise the left neighbor pixel, the right neighbor pixel, the top neighbor pixel, and the bottom neighbor pixel of the corresponding pixel in the second image.

6. The method of claim 2 , wherein pixels in the first image have more color components than the pixels in the second image.

7. The method of claim 1 wherein determining the values of the prediction parameters of the selected MMR prediction model further comprises applying numerical methods that minimize the mean square error between the first image and the output image.

8. The method of claim 3 , wherein the second order MMR model comprises a prediction model according to the formula
{circumflex over (v)} i =s i 2 {tilde over (M)} (2) +s i {tilde over (M)} (1) +n,
wherein
{circumflex over (v)} i =[{circumflex over (v)} i1 {circumflex over (v)} i2 {circumflex over (v)} i3 ] denotes the predicted three color components of the i-th pixel of the first image,
s i =[s i1 s i2 s i3 ] denotes the three color components of the i-th pixel of the second image,
s i 2 =[s i1 2 s i2 2 s i3 2 ] denotes square values of the three color components of the i-th pixel of the second image, {tilde over (M)} (1) and {tilde over (M)} (2) are 3×3 prediction parameter matrices, and n is a 1×3 prediction parameter vector, and
wherein the second order MMR model further comprises cross products according to the formula
{circumflex over (v)} i =sc i 2 {tilde over (C)} (2) +sc i 2 {tilde over (M)} (2) +sc i {tilde over (C)} (1) +s i {tilde over (M)} (1) +n,

wherein
sc i =[s i1 ·s i2 s i1 ·s i3 s i2 ·s i3 s i1 ·s i2 ·s i3 ],
sc i 2 =[s i1 2 ·s i2 2 s i1 2 ·s i3 2 s i2 2 ·s i3 2 s i1 2 ·s i2 2 ·s i3 2 ], and {tilde over (C)} (1) and {tilde over (C)} (2) comprise 4×3 prediction parameter matrices.

9. An image decoding method comprising:
receiving a first image having a first dynamic range;
receiving metadata, wherein said metadata include a multiple-regression (MMR) prediction model adapted to approximate a second image having a second dynamic range in terms of the first image, and prediction parameters of the MMR prediction model, the metadata further including previously determined values of the prediction parameters; and
applying the first image and the previously determined values of the prediction parameters to the MMR prediction model to compute an output image for approximating the second image, wherein the second dynamic range is different from the first dynamic range, and wherein pixel values of at least one color component in the output image are computed based on pixel values of at least two color components in the first image,
wherein the MMR model is a second order MMR model comprising a prediction model according to the formula:
wherein
{circumflex over (v)} i =s i 2 {tilde over (M)} (2) +s i {tilde over (M)} (1) +n,

{circumflex over (v)} i =[{circumflex over (v)} i1 {circumflex over (v)} i2 {circumflex over (v)} i3 ] denotes the predicted three color components of the i-th pixel of the second image,
s i =[s i1 s i2 s i3 ] denotes the three color components of the i-th pixel of the first image,
s i 2 =[s i1 2 s i2 2 s i3 2 ] denotes square values of the three color components of the i-th pixel of the first image, {tilde over (M)} (1) and {tilde over (M)} (2) are 3×3 prediction parameter matrices, and n is a 1×3 prediction parameter vector, and
wherein the second order MMR model further comprises cross products according to the formula:
{circumflex over (v)} i =sc i 2 {tilde over (C)} (2) +s i 2 {tilde over (M)} (2) +sc i {tilde over (C)} (1) +s i {tilde over (M)} (1) +n,

wherein
sc i =[s i1 ·s i2 s i1 ·s i3 s i2 ·s i3 s i1 ·s i2 ·s i3 ],
sc i 2 =[s i1 2 ·s i2 2 s i1 2 ·s i3 2 s i2 2 ·s i3 2 s i1 2 ·s i2 2 ·s i3 2 ], and {tilde over (C)} (1) and {tilde over (C)} (2) comprise 4×3 prediction parameter matrices.

10. The method of claim 9 wherein the first image comprises an SDR image and the second image comprises a VDR image.

11. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for executing with one or more processors a method in accordance with claim 1 .

12. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for executing with one or more processors a method in accordance with claim 9 .

13. A method to approximate using a processor an image having a first dynamic range in terms of an image having a second dynamic range, the method comprising;
receiving a first image and a second image, wherein the second image has a lower dynamic range than the first image;
selecting a multi-channel, multiple-regression (MMR) prediction model from one or more MMR models;
determining values of prediction parameters of the selected MMR model;
computing an output image approximating the first image based on the second image and the determined values of the prediction parameters of the selected MMR prediction model, wherein pixel values of at least one color component in the output image are computed based on pixel values of at least two color components in the second image; and
outputting the determi

Assignees

• Dolby Lab Licensing Corp

Inventors

• Su Guan-Ming
• Qu Sheng
• Koepfer Hubert
• Yuan Yufei
• Hulyalkar Samir

Classifications

• H04N19/30

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N19/16

  for a given display mode, e.g. for interlaced or progressive display mode · CPC title

• H04N19/103 Primary

  Selection of coding mode or of prediction mode · CPC title

• H04N19/192

  the adaptation method, adaptation tool or adaptation type being iterative or recursive · CPC title

• H04N19/98

  Adaptive-dynamic-range coding [ADRC] · CPC title