Methods and systems for demosaicing an image

Patent metadata

Field	Value
Publication number	US-9996900-B2
Application number	US-201515115906-A
Country	US
Kind code	B2
Filing date	Feb 5, 2015
Priority date	Feb 5, 2014
Publication date	Jun 12, 2018
Grant date	Jun 12, 2018

Abstract

Methods and systems are presented for converting a mosaiced image in which, for each pixel, a single color intensity value is defined for only one corresponding color, into a full color image in which, for each pixel, color intensity values are defined for each of three colors. For each color, the intensity values of the mosaiced image are interpolated, and regression filtering is performed using the intensity values of one or both of the other colors. Residual values are found as the difference between the result of the regression filtering and the mosaiced image, and these are interpolated, and added to the output of the regression, to form a full color image. The process may be iterated. In each iteration, the regression is performed using the refined channel images generated in the previous iteration.

First claim

The invention claimed is:

1. A method of processing a two-dimensional image comprising an array of pixels, the pixels being composed of (i) a first set of pixels at each of which only a ground truth red color intensity value is defined, (ii) a second set of pixels at each of which only a ground truth blue color intensity value is defined and (iii) a third set of pixels at each of which only a ground truth green color intensity value is defined, to form a two-dimensional image comprising an array of pixels at each of which a color intensity value is defined for each of red, green and blue colors;
the method including:
(a) applying at least one filter to the ground truth intensity values, to define, for each color, at least one respective interpolated color value at each of the pixels;
(b) for each of the colors, performing regression filtering using as guidance the interpolated color values for at least one of the other colors, to form an estimated color value;
(c) for each of the colors, obtaining residual values at the corresponding set of pixels as the difference between the estimated color value and the ground truth color value,
(d) for each of the colors, interpolating the residual values;
(e) for each of the colors, adding the interpolated residual values to the estimated color values to form a refined color value at each of the pixels; and
(f) repeating the set of steps (b)-(e) at least once;

in each said repetition, said step (b) being performed for each color using as guidance the refined color values for at least one of the other colors generated during the preceding performance of step (e); and wherein each time step (e) is repeated, a mean difference value is calculated indicative of the difference averaged over the image between the refined color values which step (e) produces and the refined color values produced in the previous performance of step (e), the repetition of steps (b)-(e) being terminated if the difference value is below a threshold.

2. A method according to claim 1 in which, in the first performance of step (b), the regression filtering is performed for each color on the corresponding interpolated color values.

3. A method according to claim 2 in which the generation of the red and blue channel images is not iterative.

4. A method according to claim 1 in which each said repetition of step (b) is performed on the corresponding refined color values generated during the preceding performance of step (e).

5. A method according to claim 1 , in which step (a) applies a horizontal filter and a vertical filter to the ground truth intensity values, to define, for at least one color, respective horizontally and vertically interpolated color values at each of the pixels;
steps (b)-(e) being performed separately for the horizontally and vertically interpolated color values, to generate, each of the pixels, respective said estimated color values, respective said residual values, respective said interpolated residual values, and respective said refined color values, and
the method including a step of combining the refined color values at each of the pixels to form final estimated color values.

6. A method according to claim 5 in which each final estimated color value is a weighted sum of the two refined color values respectively obtained through the horizontal and vertical processing, where the respective weights of the refined values may be different at each pixel.

7. A method according to claim 6 in which, for each of the horizontal and vertical directions, the respective weights are calculated by obtaining a measure of the smoothness of the respective set of residual values at the corresponding pixel, and setting each of the weights to be higher according to whether the smoothness of the respective set of residual values is greater than the smoothness of the other set of residual values.

8. A method according to claim 1 , including a step, prior to the repetition of steps (b)-(e), of setting a window size of at least one window used in the regression filtering.

9. A method according to claim 8 in which the window size being a function of at least one parameter t,
wherein t is set to maximise the window size subject to a constraint that an average over the pixels of a smoothness measure of the residual values within the window, is below a threshold.

10. A method according to claim 1 which is used to obtain a green channel image having green color intensity values at each of the pixels, the green channel image being used to generate red and blue channel images.

11. A system comprising a processor, and a memory device, the memory device storing program instructions operative, when performed by the processor, to cause the processor to perform a method according to claim 1 .

Assignees

• Univ Nanyang Tech

Inventors

• Ma Kai-Kuang
• Ye Wei

Classifications

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• G06T3/4007

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title