Image processing apparatus and image processing method, and program

Patent metadata

Field	Value
Publication number	US-9253459-B2
Application number	US-201214370338-A
Country	US
Kind code	B2
Filing date	Nov 29, 2012
Priority date	Jan 24, 2012
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

An apparatus for and a method of executing noise reduction processing and defect compensation processing on an image in an RGBW arrangement are provided. In pixel value compensation processing of a color pixel that makes up image data in the RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel, the W pixel is interpolated at a position of an attention pixel that is a compensation target, and at a position of a reference pixel which has the same color as an attention pixel within a reference area, smoothing weight is calculated based on each pixel value of the interpolation W pixel, and thus a compensation pixel value of the attention pixel is calculated by executing smoothing processing to which the calculated smoothing weight is applied. Moreover, by applying the W pixel in the neighborhood of the color pixel, it is determined whether or not the color pixel is in a texture area, and only if the color pixel is not in the texture, defect compensation processing is executed.

First claim

The invention claimed is:

1. An image processing apparatus comprising circuitry configured to:
receive image data acquired by an image sensor in an RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel that passes through almost all wavelength light of each wavelength of the R, G, and B,
interpolate W pixels at a position of an attention pixel that is a compensation target, and at a position of a reference pixel which has the same color as the attention pixel within a reference area, in a pixel value compensation processing of a color pixel,
calculate smoothing weight based on pixel values of the interpolation W pixels, and thus calculate a compensation pixel value of the attention pixel by executing smoothing processing to which the calculated smoothing weight is applied, wherein the smoothing processing comprises computing a product of (a) a pixel value of the reference pixel which has the same color as the attention pixel and (b) a difference between the pixel value of the interpolation W pixel at the position of the attention pixel and the pixel value of the interpolation W pixel at the position of the reference pixel, and
provide the compensation value of the attention pixel to further circuitry for use in generating a display image that corresponds to a sensed image that was sensed by the image sensor.

2. The image processing apparatus according to claim 1 ,
wherein the circuitry is further configured to determine whether or not one or more saturation pixel values are present in the pixel values of the interpolation W pixels,
wherein if the saturation pixel value is not present in the pixel values of the interpolation W pixels, the circuitry calculates a compensation pixel value of the attention pixel by executing the smoothing processing to which the smoothing weight, calculated based on the pixel values of the interpolation W pixels, is applied, and
wherein if the saturation pixel value is present in the pixel values of the interpolation W pixels, the circuitry calculates the compensation pixel value of the attention pixel by executing the smoothing processing to which the smoothing weight, calculated based on each pixel value of the attention pixel that is the compensation target, and of the reference pixel which has the same color as the attention pixel within the reference area, is applied without applying the interpolation W pixels.

3. The image processing apparatus according to claim 1 ,
wherein the circuitry is further configured to execute processing that interpolates the W pixel at the position of the reference pixel which has the same color as the attention pixel present in the reference area that is a two-dimensional area with the reference area as the two-dimensional area.

4. The image processing apparatus according to claim 1 ,
wherein the circuitry is further configured to execute processing that interpolates the W pixel at the position of the reference pixel which has the same color as the attention pixel present in the reference area that is a one-dimensional area with the reference area as the one-dimensional area.

5. The image processing apparatus according to claim 1 ,
wherein the circuitry is further configured to execute pixel value compensation as noise reduction (NR) processing that reduces noise that is included in the attention pixel.

6. The image processing apparatus according to claim 1 ,
wherein the circuitry is further configured to determine a likelihood of a defect, that is, determines whether or not there is a likelihood that the color pixel will be a defective pixel,
wherein the circuitry is further configured to execute texture detection processing that determines whether or not the color pixel is in a texture area, by applying a W pixel in the neighborhood of the color pixel that is determined as having the likelihood of the defect,
wherein in the texture detection processing, if it is determined that the color pixel is in the texture area, the circuitry does not execute defect compensation processing, and
wherein in the texture detection processing, if it is determined that the color pixel is not in the texture area, the circuitry executes the defect compensation processing.

7. The image processing apparatus according to claim 6 ,
wherein in the texture detection processing, the circuitry determines whether or not the color pixel is in the texture area, by applying a difference in the pixel value between a W pixel that is closest to the color pixel that is determined as having the likelihood of the defect, and a W pixel outside of the closest W pixel.

8. An image processing method of executing pixel value compensation in an image processing device, wherein the image processing device performs:
receiving image data acquired by an image sensor in an RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel that passes through almost all wavelength light of each wavelength of the R, G, and B;
interpolating W pixels at a position of an attention pixel that is a compensation target, and at a position of a reference pixel which has the same color as the attention pixel within a reference area, in a pixel value compensation processing of a color pixel;
calculating smoothing weight based on pixel values of the interpolation W pixels;
calculating a compensation pixel value of the attention pixel by executing smoothing processing to which the calculated smoothing weight is applied, wherein the smoothing processing comprises computing a product of (a) a pixel value of the reference pixel which has the same color as the attention pixel and (b) a difference between the pixel value of the interpolation W pixel at the position of the attention pixel and the pixel value of the interpolation W pixel at the position at the reference pixel; and
providing the compensation value of the attention pixel to circuitry for use in generating a display image that correspond to a sensed image that was sensed by the image sensor.

9. A non-transitory computer readable medium encoded with instructions which, when executed by a processor of an image processing apparatus, cause the image processing apparatus to execute pixel value compensation comprising acts of:
receiving image data acquired by an image sensor in an RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel that passes through almost all wavelength light of each wavelength of the R, G, and B;
interpolating W pixels at a position of an attention pixel that is a compensation target, and at a position of a reference pixel which has the same color as the attention pixel within a reference area, in a pixel value compensation processing of a color pixel;
calculating smoothing weight based on pixel values of the interpolation W pixels;
calculating a compensation pixel value of the attention pixel by executing smoothing processing to which the calculated smoothing weight is applied, wherein the smoothing processing comprises computing a product of (a) a pixel value of the reference pixel which has the same color as the attention pixel and (b) a difference between the pixel value of the interpolation W pixel at the position of the attention pixel and the pixel value of the interpolation W pixel at the position of the reference pixel; and
providing the compensation value of the attention pixel to circuitry for use in generating a display image that corresponds to a sensed image that was sensed by the image sensor.

10. The image processing apparatus according to claim 1 , wherein the circuitry comprises a programmed computer.

Assignees

• Sony Corp

Inventors

• Tachi Masayuki

Classifications

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• H04N2209/042

  having a single pick-up sensor · CPC title

• H04N9/646 Primary

  for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters · CPC title

• H04N2209/046

  Colour interpolation to calculate the missing colour values · CPC title

• H04N25/135

  based on four or more different wavelength filter elements · CPC title