Image processing apparatus, image processing method, and computer readable recording medium for correcting pixel value of pixel-of-interest based on correction amount calculated

Patent metadata

Field	Value
Publication number	US-10531029-B2
Application number	US-201815948000-A
Country	US
Kind code	B2
Filing date	Apr 9, 2018
Priority date	Oct 22, 2015
Publication date	Jan 7, 2020
Grant date	Jan 7, 2020

Abstract

An image processing apparatus includes: an acquisition unit configured to obtain image data generated by an image sensor forming an array pattern using color filters having mutually different spectral transmittances and to obtain a correction coefficient for correcting a difference between pixel values corresponding to a difference between a spectral sensitivity and a preset reference spectral sensitivity in a wavelength range in a pixel-of-interest; a correction amount calculation unit configured to calculate an estimation value of a color component in the pixel-of-interest using a pixel value of each of pixels in surrounding pixels of a same color and the correction coefficient and configured to calculate a correction amount of the pixel value of the pixel-of-interest based on the estimation value and the correction coefficient of the pixel-of-interest; and a pixel value correction unit configured to correct the pixel value of the pixel-of-interest based on the calculated correction amount.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a processor comprising hardware, wherein the processor is configured to:
obtain image data generated by an image sensor that forms a predetermined array pattern using color filters of a plurality of colors having mutually different spectral transmittances and in which each of the color filters is arranged at a position corresponding to any of a plurality of pixels;
obtain a correction coefficient for each of the plurality of pixels, the correction coefficient for correcting a difference between pixel values corresponding to a difference between a spectral sensitivity and a preset reference spectral sensitivity in a predetermined wavelength range in a pixel-of-interest;
calculate an estimation value of a color component as a correction target in the pixel-of-interest using a pixel value of each of pixels in surrounding pixels of a same color and the correction coefficient, the surrounding pixel of a same color being a pixel surrounding the pixel-of-interest and being a color filter arranged on the each of the surrounding pixels and a pixel including a color filter having a same color as a color filter arranged on the pixel-of-interest;
calculate a correction amount of the pixel value of the pixel-of-interest based on the estimation value and the correction coefficient of the pixel-of-interest; and
correct the pixel value of the pixel-of-interest based on the correction amount calculated.

2. The image processing apparatus according to claim 1 ,
wherein the processor is configured to:
calculate a similarity level from a theoretical value of the pixel value in the surrounding pixel of a same color calculated based on a candidate value and the correction coefficient of each of the pixels in the surrounding pixel of a same color and from a measured value of the pixel value in the surrounding pixel of a same color; and
calculate the candidate value having a high similarity level as the estimation value.

3. The image processing apparatus according to claim 2 ,
wherein the theoretical value is a product of the candidate value and the correction coefficient of each of the pixels in the surrounding pixels of a same color.

4. The image processing apparatus according to claim 2 ,
wherein the similarity level is a value based on a difference between the theoretical value and the measured value of each of the pixels in the surrounding pixels of a same color.

5. The image processing apparatus according to claim 1 ,
wherein the processor is configured to:
divide a pixel value of each of the pixels in the surrounding pixels of a same color by the correction coefficient; and
calculate the estimation value based on the value obtained by the division.

6. The image processing apparatus according to claim 5 ,
wherein the processor is configured to calculate a statistical value obtained by dividing the pixel value of each of the pixels in the surrounding pixels of a same color by the correction coefficient as the estimation value.

7. The image processing apparatus according to claim 1 ,
wherein the processor is configured to calculate, for the surrounding pixel of a same color, the estimation value by solving simultaneous equations including a relational expression between the pixel value in the surrounding pixel of a same color, the correction coefficient, and the estimation value, established for each of the pixels.

8. The image processing apparatus according to claim 7 ,
wherein the processor is configured to calculate, as the estimation value, a statistical value of the estimation value calculated by combining each of a plurality of pixels of the surrounding pixels of a same color.

9. The image processing apparatus according to claim 1 ,
wherein the processor is configured to calculate, in surrounding portions of the pixel-of-interest, the estimation value based on a ratio of a pixel value of each of the pixels in the surrounding pixels in which pixel values are not saturated to the correction coefficient of each of the pixels in the surrounding pixels of a same color.

10. The image processing apparatus according to claim 1 ,
wherein the processor is configured to correct the pixel value of the pixel-of-interest using the correction amount such that the corrected pixel value of the pixel-of-interest falls from an average value of the pixel value of each of the pixels in the surrounding pixels of a same color to the pixel value of the pixel-of-interest.

11. The image processing apparatus according to claim 1 ,
wherein the processor is configured to not subtract the correction amount from the pixel value of the pixel-of-interest in a case where a difference between the value obtained by subtracting the correction amount from the pixel value of the pixel-of-interest and the average value of the pixel value of the surrounding pixels of a same color is larger than a predetermined value.

12. An image processing method comprising:
obtaining image data generated by an image sensor that forms a predetermined array pattern using color filters of a plurality of colors having mutually different spectral transmittances and in which each of the color filters is arranged at a position corresponding to any of a plurality of pixels and obtaining a correction coefficient from a recording unit configured to record, for each of the pixels, the correction coefficient for correcting a difference between pixel values corresponding to a difference between a spectral sensitivity and a preset reference spectral sensitivity in a predetermined wavelength range in a pixel-of-interest;
calculating an estimation value of a color component as a correction target in the pixel-of-interest using a pixel value of each of pixels in a surrounding pixel of a same color and the correction coefficient, the surrounding pixel of a same color being a pixel surrounding the pixel-of-interest and being a pixel including a color filter having a same color as a color filter arranged on the pixel-of-interest;
calculating a correction amount of the pixel value of the pixel-of-interest based on the estimation value and the correction coefficient of the calculated pixel-of-interest; and
correcting the pixel value of the pixel-of-interest based on the calculated correction amount.

13. A non-transitory computer-readable recording medium on which an executable program is recorded, the program instructing a processor of an image processing apparatus to execute:
obtaining image data generated by an image sensor that forms a predetermined array pattern using color filters of a plurality of colors having mutually different spectral transmittances and in which each of the color filters is arranged at a position corresponding to any of a plurality of pixels and obtaining a correction coefficient from a recording unit configured to record, for each of the pixels, the correction coefficient for correcting a difference between pixel values corresponding to a difference between a spectral sensitivity and a preset reference spectral sensitivity in a predetermined wavelength range in a pixel-of-interest;
calculating an estimation value of a color component as a correction target in the pixel-of-interest using a pixel value of each of pixels in a surrounding pixel of a same color and the correction coefficient, the surrounding pixel of a same color being a pixel surrounding the pixel-of-interest and being a pixel including a color filter having a same color as a color filter arranged on the pixel-of-interest;
calculating a correction amount of the pixel value of the pixel-of-interest based on the estimation value and the correction coefficient of

Assignees

• Olympus Corp

Inventors

• Maruyama Hiroki
• Ichikawa Manabu

Classifications

• H04N25/683 Primary

  by defect estimation performed on the scene signal, e.g. real time or on the fly detection · CPC title

• H04N23/12

  with one sensor only · CPC title

• H04N23/84 Primary

  for processing colour signals · CPC title

• G01J3/28

  Investigating the spectrum (using colour filters G01J3/51) · CPC title

• H04N2209/046

  Colour interpolation to calculate the missing colour values · CPC title