Image processing apparatus, imaging device, image processing method, and program for reducing noise or false colors in an image

Patent metadata

Field	Value
Publication number	US-9699429-B2
Application number	US-201214384835-A
Country	US
Kind code	B2
Filing date	Dec 26, 2012
Priority date	Mar 27, 2012
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

There are provided an apparatus and a method which generate an RGB image having less color noise and fewer false colors by inputting an RGBW image. The apparatus has an image sensor having an RGBW array, and an image processing unit which performs image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor. The image sensor has a periodic array of a unit composition formed of each RGBW pixel, and has an array in which composition ratios of each RGB pixel within the unit composition are adapted to be the same as each other. The image processing unit converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs at least either array conversion processing for generating an RGB array image or signal processing for generating each RGB image signal in which all RGB pixel values are set for each pixel position of the sensor image.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
an image sensor that has an RGBW array including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light; and
an image processing unit that performs image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor,
wherein the image sensor has a periodic array of a unit composition formed of each RGBW pixel, and has an array in which composition ratios of the R, G and B pixels within the unit composition are the same as each other and wherein each row and column of the periodic array are configured so that one W pixel is interposed between each of the R, G and B pixels and the R, G and B pixels are sequentially arranged in each row and column, and
wherein the image processing unit converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs at least either array conversion processing for generating an RGB array image or signal processing for generating each RGB image signal in which all RGB pixel values are set for each pixel position of the sensor image.

2. The image processing apparatus according to claim 1 ,
wherein the image processing unit converts the pixel array of the sensor image formed of the RGBW pixel signal, and has an array conversion unit which generates the RGB array image formed of a Bayer array, and
wherein the array conversion unit has a W pixel interpolation unit which generates a whole W pixel image signal by setting the W pixel at an RGB pixel position of the sensor image; an RGBW pixel generation unit which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and a correlation processing unit which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal, and which performs array conversion for a composition pixel of the sensor image and generates an RGB image of the Bayer array by calculating a pixel value based on an estimation in which the W pixel and the R, G and B pixels have a positive correlation.

3. The image processing apparatus according to claim 2 ,
wherein the W pixel interpolation unit detects an edge direction by applying a pixel value of the W pixel of the sensor image, and determines a W pixel value at an interpolation pixel position by regarding a pixel present in the detected edge direction as a reference pixel.

4. The image processing apparatus according to claim 1 ,
wherein the image processing unit has a demosaic processing unit which performs demosaic processing on the sensor image and generates each RGB image signal in which all RGB pixel values are set for each pixel position of the sensor image, and
wherein the demosaic processing unit has a W pixel interpolation unit which generates a whole W pixel image signal by setting the W pixel at an RGB pixel position of the sensor image; an RGBW pixel generation unit which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and a correlation processing unit which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal, and which generates each RGB pixel signal in which all RGB pixel values are set for each pixel position of the sensor image, by calculating a pixel value based on an estimation in which the W pixel and the RGB pixel have a positive correlation.

5. The image processing apparatus according to claim 4 ,
wherein the W pixel interpolation unit detects an edge direction by applying a pixel value of the W pixel of the sensor image, and determines a W pixel value at an interpolation pixel position by regarding a pixel present in the detected edge direction as a reference pixel.

6. The image processing apparatus according to claim 4 ,
wherein the image processing unit has a linear matrix processing unit which performs pixel value conversion to which a preset matrix is applied, on each RGB image signal generated by the demosaic processing unit.

7. The image processing apparatus according to claim 1 ,
wherein the image processing unit has a demosaic processing unit which performs demosaic processing on the sensor image and generates each RGBW image signal in which all RGB and W pixel values are set for each pixel position of the sensor image, and
wherein the demosaic processing unit has a W pixel interpolation unit which generates a whole W pixel image signal by setting the W pixel at an RGB pixel position of the sensor image; an RGBW pixel generation unit which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and a correlation processing unit which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal, and which generates each RGB pixel signal in which all RGB pixel values are set for each pixel position of the sensor image, by calculating a pixel value based on an estimation in which the W pixel and the RGB pixel have a positive correlation.

8. The image processing apparatus according to claim 7 ,
wherein the image processing unit has a linear matrix processing unit which performs pixel value conversion to which a preset matrix is applied, on each RGBW image signal generated by the demosaic processing unit.

9. The image processing apparatus according to claim 1 ,
wherein the image sensor is configured to output an image having an added pixel value of the same color pixel values which are adjacent to each other in an oblique direction, as the sensor image,
wherein the image processing unit has an array conversion unit which converts a pixel array of the sensor image configured to have the added pixel value and formed of the RGBW pixel signal, and which generates an RGB array image formed of a Bayer array, and
wherein the array conversion unit has an RGBW pixel generation unit which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and a correlation processing unit which inputs the whole W pixel image signal output as the sensor image and the RGBW corresponding low frequency signal, and which performs array conversion on a composition pixel of the sensor image and generates an RGB image of the Bayer array, by calculating a pixel value based on an estimation in which the W pixel and the RGB pixel have a positive correlation.

10. The image processing apparatus according to claim 1 ,
wherein the image sensor is configured to output an image including a long-period exposure pixel and a short-period exposure pixel which are set according to a control of a control unit, as the sensor image,
wherein the image processing unit has an array conversion unit which generates a high dynamic range (HDR) image by applying the sensor image including the long-period exposure pixel and the short-period exposure pixel, and which generates an RGB array HDR image formed of a Bayer array by converting the pixel array, and
wherein the array conversion unit has a W pixel interpolation unit which generates a whole W pixel long-period exposure image in which a long-period exposure W pixel is set for all pixel positions, by applying a long-period exposure pixel signal of the sensor image, and which generates a whole W pixel short-period exposure image in which a short-period exposure W pixel is set for all pixel positions, by applying a short-period exposure

Assignees

• Sony Corp

Inventors

• Kaizu Shun

Classifications

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N25/135

  based on four or more different wavelength filter elements · CPC title

• H04N23/12

  with one sensor only · CPC title

• H04N25/589

  with different integration times, e.g. short and long exposures · CPC title

• H04N23/741 Primary

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title