Image generation method, image generation apparatus, program, and storage medium

Patent metadata

Field	Value
Publication number	US-9609208-B2
Application number	US-201615130367-A
Country	US
Kind code	B2
Filing date	Apr 15, 2016
Priority date	Jan 13, 2012
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

An image generation method is provided for generating an output image from an input image acquired by an image sensor that has an array of multiple pixels, each of which has arranged therein multiple sub-pixels that each receive a light beam that passes through a different pupil sub-area of an imaging optical system. The method includes a step of generating multiple parallax images that respectively correspond to the different pupil sub-areas; a step of generating multiple pixel shifted images by performing different shifting for each of the parallax images according to a virtual image forming plane of the imaging optical system; and a step of generating an output image that has a higher resolution than the resolution of the parallax images from the pixel shifted images through composition processing.

First claim

The invention claimed is:

1. An image generation method for generating an output image from a plurality of input images captured at a plurality of viewpoints by an image sensor, the method comprising:
a step of generating a plurality of pixel shifted images by performing different shifting for each of the plurality of input images according to a virtual image forming plane that is different from an image sensing plane at which the input images are captured, wherein an amount of pixel shifting is non-integral; and
a step of generating an output image that has a higher resolution than the resolution of the input images from the plurality of pixel shifted images through composition processing.

2. The image generation method according to claim 1 , wherein letting Np be the number of different viewpoints, F be an aperture value of an imaging optical system, and Δx be a period of sub-pixels of the input images, the distance between the image sensing plane and the virtual image forming plane is less than or equal to NpFΔx.

3. The image generation method according to claim 1 , wherein letting F be an aperture value of an imaging optical system, and δ be the diameter of a permissible circle of confusion, the distance between the image sensing plane and the virtual image forming plane is less than or equal to a focal depth Fδ.

4. The image generation method according to claim 2 , wherein an amount of shift in the shifting for each of the plurality of input images is Δx.

5. A non-transitory computer-readable storage medium that stores a program for causing a computer to execute the steps of the image generation method according to claim 1 .

6. An image generation apparatus comprising:
one or more processors configured to:
input a plurality of input images captured at a plurality of viewpoints by an image sensor;
generate a plurality of pixel shifted images by performing different shifting for each of the plurality of input images according to a virtual image forming plane that is different from an image sensing plane at which the input images are captured, wherein an amount of pixel shifting is non-integral; and
generate an output image that has a higher resolution than the resolution of the input images from the plurality of pixel shifted images through composition processing.

7. The image generation apparatus according to claim 6 , wherein letting Np be the number of different viewpoints, F be an aperture value of an imaging optical system, and Δx be a period of sub-pixels of the input images, the distance between the image sensing plane and the virtual image forming plane is less than or equal to NpFΔx.

8. The image generation apparatus according to claim 6 , wherein letting F be an aperture value of an imaging optical system, and δ be the diameter of a permissible circle of confusion, the distance between the image sensing plane and the virtual image forming plane is less than or equal to a focal depth Fδ.

9. The image generation apparatus according to claim 7 , wherein an amount of shift in the shifting for each of the plurality of input images is Δx.

Assignees

• Canon Kk

Inventors

• Fukuda Koichi
• Kimura Masafumi
• Hatakeyama Koshi
• Hiasa Norihito
• Tsutsumi Shohei
• Nishiyama Tomohiro
• Yahata Kazuhiro

Classifications

• H04N23/951 Primary

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• G02B7/346 Primary

  using horizontal and vertical areas in the pupil plane, i.e. wide area autofocusing · CPC title