Image processing apparatus, image processing method, and program for generating a three dimensional image to be stereoscopically viewed

Patent metadata

Field	Value
Publication number	US-9118904-B2
Application number	US-201414564234-A
Country	US
Kind code	B2
Filing date	Dec 9, 2014
Priority date	Aug 4, 2011
Publication date	Aug 25, 2015
Grant date	Aug 25, 2015

Abstract

There is provided an imaging element that photographs multiple viewing point images corresponding to images observed from different viewing points and an image processing unit separates an output signal of the imaging element, acquires the plurality of viewing point images corresponding to the images observed from the different viewing points, and generates a left eye image and a right eye image for three-dimensional image display, on the basis of the plurality of acquired viewing point images. The image processing unit generates parallax information on the basis of the plurality of viewing point images obtained from the imaging element and generates a left eye image and a right eye image for three-dimensional image display by 2D3D conversion processing using the generated parallax information. By this configuration, a plurality of viewing point images are acquired on the basis of one photographed image and images for three-dimensional image display are generated.

First claim

What is claimed is:

1. An image processing apparatus, comprising:
an image processing unit configured to execute signal processing for an output signal of an imaging element, the image processing unit comprising:
a parallax detecting unit configured to apply two different viewing point images obtained from the imaging element, and to generate parallax information, wherein the parallax detecting unit is further configured such that the two different viewing point images are obtained from output signals from pixels forming the imaging element corresponding to a microlens arrangement region; and
an image correcting unit configured to execute correction processing for a central viewing point image from the imaging element and to generate a two-dimensional image, and wherein the image correcting unit is further configured such that the central viewing point image is obtained from output signals from pixels forming the imaging element corresponding to a microlens non-setting region.

2. The image processing apparatus according to claim 1 , wherein the image processing unit further comprises:
a parallax image generating unit configured to apply the parallax information generated by the parallax detecting unit, and to execute image conversion processing of the two-dimensional image generated by the image correcting unit.

3. The image processing apparatus according to claim 1 , further comprising:
the imaging element; and
an incident light control unit comprising a microlens array in which microlenses are arranged on a part of a region of the imaging element, the incident light control unit being configured to perform incident light control to cause the imaging element to photograph a plurality of viewing point images corresponding to images observed from different viewing points, and wherein the incident light control unit has an incident light control configuration to acquire three viewing point images including a left eye viewing point image, a right eye viewing point image, and the central viewing point image.

4. The image processing apparatus according to claim 3 ,
wherein the incident light control unit includes:
a first polarizing unit that has a first polarization region and a second polarization region to transmit different polarized light; and
a second polarizing unit configured to cause transmission light of the first polarizing unit to be incident and that includes a third polarization region to transmit only the transmission light of the first polarization region, a fourth polarization region to transmit only the transmission light of the second polarization region, and a total transmission region to transmit the transmission light of the first polarization region and the second polarization region, and
wherein the image processing unit is further configured such that the two different viewing point images include an image based on the transmission light of the third polarization region of the second polarizing unit and an image based on the transmission light of the fourth polarization region of the second polarizing unit.

5. The image processing apparatus according to claim 4 ,
wherein the third polarization region and the fourth polarization region of the second polarizing unit are polarization regions that are configured using a wire grid polarizer.

6. The image processing apparatus according to claim 3 , wherein:
the incident light control unit further includes a light shielding film that is set to an arrangement region of the microlenses and shields a part of incident light with respect to the imaging element;
the parallax detecting unit is further configured such that the two different viewing point images are obtained from output signals from pixels forming the imaging element corresponding to the arrangement region of the microlenses and the light shielding film;
the image correcting unit is further configured to execute correction processing with respect to an image obtained from output signals of pixels forming the imaging element corresponding to a non-setting region of the microlenses and the light shielding film.

7. The image processing apparatus according to claim 3 ,
wherein the imaging element includes:
a left/right viewing point image acquisition region to acquire two different viewing point images to generate parallax information in which an object distance is reflected; and
a central viewing point image acquisition region to generate an image photographed from one viewing point.

8. The image processing apparatus according to claim 7 ,
wherein the left/right viewing point image acquisition region is configured using pixels of one color, and
the image processing unit has a parallax detecting unit configured to apply two different viewing point images obtained from output signals of the left/right viewing point image acquisition region configured using the pixels of one color.

9. The image processing apparatus according to claim 3 ,
wherein the incident light control unit includes:
a first polarizing unit that has a first polarization region and a second polarization region to transmit different polarized light; and
a second polarizing unit that causes transmission light of the first polarizing unit to be incident and includes a third polarization region to transmit only the transmission light of the first polarization region and a fourth polarization region to transmit only the transmission light of the second polarization region, and
wherein the image processing unit is further configured to execute interpolation processing for setting pixel values to defective pixel regions of two different viewing point images of an image based on transmission light of the third polarization region of the second polarizing unit and an image based on transmission light of the fourth polarization region of the second polarizing unit.

10. The image processing apparatus according to claim 9 ,
wherein the third polarization region and the fourth polarization region of the second polarizing unit are polarization regions that are configured using a wire grid polarizer.

11. The image processing apparatus according to claim 1 ,
wherein the image processing unit is further configured to generate a depth map in which object distance information of images is reflected as parallax information in which an object distance is reflected, and to generate a left eye image and a right eye image for three-dimensional image display by image conversion processing to which the depth map is applied.

12. The image processing apparatus according to claim 1 ,
wherein the parallax image generating unit is further configured to execute 2D3D conversion processing for generating a left eye image and a right eye image for three-dimensional image display by image conversion processing to which the parallax information is applied, for the two-dimensional image generated by the image correcting unit.

13. The image processing apparatus according to claim 12 ,
wherein the 2D3D conversion processing that is executed by the image processing unit is 2D3D conversion processing for executing image shift processing according to the object distance for the first viewing point images and generating the left eye image and the right eye image for three-dimensional image display.

14. An image processing method of executing image processing by an image processing apparatus, comprising:
separating an output signal from an imaging element into a plurality of viewing point images corresponding to images observed from different viewing points by separation processing of the output signal from the imaging element, the plurality of viewing point ima

Assignees

• Sony Corp

Inventors

• Kobayashi Seiji
• Ito Atsushi

Classifications

• G02B30/25 Primary

  using polarisation techniques · CPC title

• H04N13/225

  using parallax barriers · CPC title

• H04N13/261

  with monoscopic-to-stereoscopic image conversion · CPC title

• H04N13/296

  Synchronisation thereof; Control thereof · CPC title

• H04N13/218 Primary

  using spatial multiplexing · CPC title