Image processing apparatus and image processing program

Patent metadata

Field	Value
Publication number	US-9706190-B2
Application number	US-201314047269-A
Country	US
Kind code	B2
Filing date	Oct 7, 2013
Priority date	Apr 14, 2011
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

So as to obtain color parallax images, it has been necessary to prepare an image capturing optical system and an image capturing element that are complex, to capture the parallax images. In view of the above, provided is an image processing apparatus including: an image obtaining section; a disparity amount operation section; and an image converting section.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a processor configured to perform operations including:
obtaining reference image data and parallax image data generated by image-capturing the same scene, the parallax image data having parallax with respect to a subject image of the reference image data and including right parallax image data that is shifted to a right direction with respect to the reference image data and left parallax image data that is shifted to a left direction with respect to the reference image data,
calculating an amount of disparity that corresponds to parallax between the same subject images respectively of the reference image data and the parallax image data, and
generating image data by implanting, based on the amount of disparity, at least a part of primary color information for generating a color image of the reference image data to different locations within an image for the right parallax image data and the left parallax image data.

2. The image processing apparatus according to claim 1 , wherein the processor is further configured to:
obtain an image data group including the reference image data and the parallax image data, at least one of the reference image data and the parallax image data being non-color image data lacking at least a part of the primary color information, and
convert the non-color image data to color image data.

3. The image processing apparatus according to claim 2 , wherein
the first parallax image data and the second parallax image data are the non-color image data, and
the processor is further configured to implant at least a part of the primary color information of the reference image data to the first parallax image data and the second parallax image data, thereby converting the first parallax image data and the second parallax image data to the color image data.

4. The image processing apparatus according to claim 3 , wherein
the first parallax image data and the second parallax image data are brightness image data that does not have the primary color information.

5. The image processing apparatus according to claim 2 , wherein
the reference image data is the non-color image data, and
the processor is further configured to implant at least a part of the primary color information of at least one of the first parallax image data and the second parallax image data to the reference image data, thereby converting the reference image data to the color image data.

6. The image processing apparatus according to claim 2 , wherein
the reference image data, the first parallax image data, and the second parallax image data are the non-color image data, and
the processor is further configured to implant at least a part of the primary color information of at least one of the first parallax image data and the second parallax image data to the reference image data, and implant at least a part of the primary color information of the reference image data to the first parallax image data and the second parallax image data, thereby converting the reference image data, the first parallax image data, and the second parallax image data to the color image data.

7. The image processing apparatus according to claim 2 , wherein the processor is further configured to
calculate a disparity amount between the reference image data and the first parallax image data, and a disparity amount between the reference image data and the second parallax image data, after calculating the disparity amount of the first parallax image data and the second parallax image data.

8. The image processing apparatus according to claim 2 , wherein the processor is further configured to
generate a disparity map representing the disparity amount of the first parallax image data and the second parallax image data with respect to the reference image data, and
implant the primary color information by referring to the disparity map.

9. An image capturing apparatus comprising:
an image capturing element that image-captures the same scene; and
an image processing apparatus according to claim 1 , wherein
the processor is further configured to obtain the reference image data and the parallax image data by generating the reference image data and the parallax image data from a signal outputted from the image capturing element.

10. The image capturing apparatus according to claim 9 , wherein
the image capturing element includes:
photoelectric conversion elements arranged two dimensionally, and photoelectric converting incident light to an electric signal;
opening masks provided in a one-to-one relation to at least a part of the photoelectric conversion elements; and
color filters provided in a one-to-one relation to at least a part of the photoelectric conversion elements, wherein
openings of the opening masks provided to correspond to at least two of n (n being an integer equal to or greater than 3) photoelectric conversion elements adjacent to each other are included in one of color filter patterns including at least three types of the color filters transmitting respectively different wavelength regions, and are positioned to respectively transmit light fluxes from partial regions mutually different within a sectional region of the incident light, and the groups of photoelectric conversion elements, each made up of a set of n photoelectric conversion elements, are periodically arranged.

11. The image processing apparatus according to claim 1 , wherein the processor is further configured to define, based on the amount of disparity, a location of an implanted area within an image when implanting the at least a part of primary color information.

12. The image processing apparatus according to claim 1 , wherein the reference image data is image data without parallax.

13. An image processing apparatus comprising:
a processor configured to perform operations including:
obtaining an image data group including at least (i) reference image data generated by image capturing a scene, (ii) first parallax image data that is generated by image-capturing the same scene and that has the parallax in one direction with respect to the reference image data, and (iii) second parallax image data that is generated by image-capturing the same scene and that has parallax in another direction that is reverse to the one direction, the first and second parallax image data having parallax with respect to a subject image of the reference image data, and at least one of (i) the reference image data and (ii) the first and second parallax image data being non-color image data lacking at least a part of primary color information for generating a color image,
calculating disparity amounts that correspond to parallax between the same subject images respectively of the reference image data and the first and second parallax image data by, after calculating the disparity amount of the first parallax image data and the second parallax image data, calculating (i) the disparity amount between the reference image data and the first parallax image data and (ii) the disparity amount between the reference image data and the second parallax image data, and
converting the non-color image data to color image data by implanting at least a part of primary color information for generating a color image of one of the reference image data and the first and second parallax image data to the other of the reference image data and the first and second parallax image data based on the disparity amounts.

Assignees

• Nikon Corp

Inventors

• Shibazaki Kiyoshige
• Hamashima Muneki
• Mori Susumu
• Suzuki Yumi

Classifications

• H04N23/6842

  by controlling the scanning position, e.g. windowing · CPC title

• H04N25/702

  SSIS architectures characterised by non-identical, non-equidistant or non-planar pixel layout · CPC title

• H04N25/133

  including elements passing panchromatic light, e.g. filters passing white light · CPC title

• H04N23/672

  based on the phase difference signals · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title