Image processing apparatus and image processing method, and program

Patent metadata

Field	Value
Publication number	US-9445022-B2
Application number	US-201514982239-A
Country	US
Kind code	B2
Filing date	Dec 29, 2015
Priority date	Jan 24, 2012
Publication date	Sep 13, 2016
Grant date	Sep 13, 2016

Abstract

An apparatus for and a method of executing noise reduction processing and defect compensation processing on an image in an RGBW arrangement are provided. In pixel value compensation processing of a color pixel that makes up image data in the RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel, the W pixel is interpolated at a position of an attention pixel that is a compensation target, and at a position of a reference pixel which has the same color as an attention pixel within a reference area, smoothing weight is calculated based on each pixel value of the interpolation W pixel, and thus a compensation pixel value of the attention pixel is calculated by executing smoothing processing to which the calculated smoothing weight is applied. Moreover, by applying the W pixel in the neighborhood of the color pixel, it is determined whether or not the color pixel is in a texture area, and only if the color pixel is not in the texture, defect compensation processing is executed.

First claim

The invention claimed is:

1. An image processing apparatus comprising circuitry configured to:
receive image data acquired by an image sensor in an RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel,
interpolate W pixels at a plurality of positions including a position of a noise-reduction target pixel and at positions of a plurality of reference pixels which have the same color as the noise-reduction target pixel,
calculate a first smoothing weight based on pixel values of the interpolated W pixels at the plurality of positions,
determine whether or not any saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, and
calculate a pixel value of the noise-reduction target pixel by:
if no saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, calculating the pixel value of the noise-reduction target pixel by executing first smoothing processing to which the first smoothing weight is applied, and
if one or more saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, calculating the pixel value of the noise-reduction target pixel by executing second smoothing processing to which a second smoothing weight is applied, the second smoothing weight being calculated based on a pixel value of the noise-reduction target pixel and on pixel values of the plurality of reference pixels which have the same color as the noise-reduction target pixel.

2. The image processing apparatus of claim 1 , wherein the circuitry is further configured to receive the image data acquired by the image sensor in which the plurality of reference pixels which have the same color as the noise-reduction target pixel are included in a two-dimensional area surrounding the noise reduction target pixel and are each adjacent at least two W pixels.

3. The image processing apparatus of claim 2 , wherein the circuitry is further configured to receive the image data acquired by the image sensor in which the plurality of reference pixels which have the same color as the noise-reduction target pixel are each adjacent at least four W pixels.

4. The image processing apparatus of claim 1 , wherein the circuitry is further configured to receive the image data acquired by the image sensor in which the plurality of reference pixels which have the same color as the noise-reduction target pixel are included in a one-dimensional area surrounding the noise reduction target pixel and are each adjacent at least two W pixels.

5. The image processing apparatus of claim 1 , wherein the circuitry is further configured to execute pixel value compensation as noise reduction (NR) processing that reduces noise that is included in the attention pixel.

6. The image processing apparatus of claim 1 , wherein the circuitry is further configured to:
determine whether or not one of the color pixels is likely to be a defective pixel,
if it is determined that the one of the color pixels is likely to be a defective pixel, execute texture detection processing that determines whether or not the one of the color pixels is in a texture area, by applying a W pixel in a neighborhood of the one of the color pixels, and
execute defect compensation processing if it is determined that the color pixel is not in the texture area, and refrain from executing defect compensation processing if it is determined that the color pixel is in the texture area.

7. The image processing apparatus of claim 6 , wherein the circuitry is further configured execute the detection processing by applying a difference in the pixel value between a W pixel that is closest to the one of the color pixels that is determined likely to be a defective pixel, and a W pixel outside of the closest W pixel.

8. The image processing apparatus according to claim 1 , wherein the circuitry comprises a programmed computer.

9. An image processing method of executing pixel value compensation in an image processing device, comprising:
receiving image data acquired by an image sensor in an RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel,
interpolating W pixels at a plurality of positions including a position of a noise-reduction target pixel and at positions of a plurality of reference pixels which have the same color as the noise-reduction target pixel,
calculating a first smoothing weight based on pixel values of the interpolated W pixels at the plurality of positions,
determining whether or not any saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, and
calculating a pixel value of the noise-reduction target pixel by:
if no saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, calculating the pixel value of the noise-reduction target pixel by executing first smoothing processing to which the first smoothing weight is applied, and
if one or more saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, calculating the pixel value of the noise-reduction target pixel by executing second smoothing processing to which a second smoothing weight is applied, the second smoothing weight being calculated based on a pixel value of the noise-reduction target pixel and on pixel values of the plurality of reference pixels which have the same color as the noise-reduction target pixel.

10. A non-transitory computer readable medium encoded with instructions which, when executed by a processor of an image processing apparatus, cause the image processing apparatus to perform acts of:
receiving image data acquired by an image sensor in an RGBW arrangement that has each color pixel of R, G, and B and a white (W) pixel,
interpolating W pixels at a plurality of positions including a position of a noise-reduction target pixel and at positions of a plurality of reference pixels which have the same color as the noise-reduction target pixel,
calculating a first smoothing weight based on pixel values of the interpolated W pixels at the plurality of positions,
determining whether or not any saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, and
calculating a pixel value of the noise-reduction target pixel by:
if no saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, calculating the pixel value of the noise-reduction target pixel by executing first smoothing processing to which the first smoothing weight is applied, and
if one or more saturation pixel values are present in the pixel values of the interpolated W pixels at the plurality of positions, calculating the pixel value of the noise-reduction target pixel by executing second smoothing processing to which a second smoothing weight is applied, the second smoothing weight being calculated based on a pixel value of the noise-reduction target pixel and on pixel values of the plurality of reference pixels which have the same color as the noise-reduction target pixel.

Assignees

• Sony Corp

Inventors

• Tachi Masayuki

Classifications

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• H04N9/646

  for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters · CPC title

• H04N2209/046

  Colour interpolation to calculate the missing colour values · CPC title

• H04N2209/042

  having a single pick-up sensor · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title