Image processing device and method, and imaging device

Patent metadata

Field	Value
Publication number	US-9277201-B2
Application number	US-201414499933-A
Country	US
Kind code	B2
Filing date	Sep 29, 2014
Priority date	Mar 30, 2012
Publication date	Mar 1, 2016
Grant date	Mar 1, 2016

Abstract

An image processing method according to the present invention acquires a first image and second image that are picked up through a single image-taking optical system, that are images after a pupil division, and that have a parallax to each other, and then performs a filtering process for each pixel of the first and second images, using first and second transform filters that correspond to the parallax for the pixel and that are of first and second transform filter groups to be respectively applied to the first and second images for the transform into third and fourth images in which the parallax amount and blur amount of the first and second images have been altered.

First claim

What is claimed is:

1. An image processing device comprising:
an image acquisition device for acquiring a first image and a second image that are picked up through a single image-taking optical system and that have a parallax to each other, the first image and the second image being images after a pupil division by a pupil-division device;
a parallax acquisition device for acquiring a parallax between corresponding pixels of the acquired first image and second image; and
a filtering processing device for performing a filtering process for each pixel of the acquired first image and second image, using a first transform filter and a second transform filter corresponding to the parallax for the pixel acquired by the parallax acquisition device, the first transform filter and the second transform filter being of a first transform filter group and a second transform filter group for transforming into a third image and a fourth image in which a parallax amount and a blur amount of the acquired first image and second image have been altered, the first transform filter group being applied to the first image, the second transform filter group being applied to the second image.

2. The image processing device according to claim 1 , wherein the filtering processing device increases the blur amount, as the parallax amount between corresponding pixels of the third image and the fourth image increases.

3. The image processing device according to claim 1 , comprising:
a digital filter storage device for storing a first digital filter group and a second digital filter group for each parallax, the first digital filter group and the second digital filter group corresponding to a transform characteristic by which light entering the single image-taking optical system is transformed into the first image and the second image; and
a transform filter calculation device for calculating the first transform filter group and the second transform filter group, based on the stored first digital filter group and second digital filter group, and a third digital filter group and a fourth digital filter group for each pixel, the third digital filter group and the fourth digital filter group corresponding to a transform characteristic by which the light entering the single image-taking optical system is transformed into the third image and the fourth image,
wherein, for each pixel of the acquired first image and second image, based on the parallax for the pixel acquired by the parallax acquisition device, the filtering processing device selects the first transform filter and second transform filter corresponding to the parallax, from the calculated first transform filter group and second transform filter group, and then performs the filtering process using the selected first transform filter and second transform filter.

4. The image processing device according to claim 3 , wherein the transform filter calculation device calculates a transform filter to be convoluted in real space, by an amplitude component of the following formula,
F −1 (T f (ω x , ω y ) −1 ·T wf (ω x , ω y ))

(here, F −1 : Fourier inverse transform),
when any one digital filter of the first digital filter group and the second digital filter group stored in the digital filter storage device is T(x, y), the digital filter T(x, y) to which Fourier transform has been performed is T f (ω x , ω y ), a digital filter that is of the third digital filter group and the fourth digital filter group and that corresponds to the digital filter T(x, y) is T w (x, y), and the digital filter T w (x, y) to which Fourier transform has been performed is T wf (ω x , ω y ).

5. The image processing device according to claim 3 , comprising:
a specification device for specifying a transform characteristic for the third image and the fourth image that are to be generated by the transform; and
a digital filter calculation device for calculating the third digital filter group and fourth digital filter group corresponding to the specified transform characteristic,
wherein the transform filter calculation device calculates the first transform filter group and the second transform filter group, using the first digital filter group and the second digital filter group stored in the digital filter storage device, and the third digital filter group and the fourth digital filter group calculated by the digital filter calculation device.

6. The image processing device according to claim 1 , comprising a transform filter storage device for storing the first transform filter group and the second transform filter group for each parallax that are calculated based on a first digital filter group and a second digital filter group for each parallax and a third digital filter group and a fourth digital filter group for each parallax, the first digital filter group and the second digital filter group corresponding to a transform characteristic by which light entering the single image-taking optical system is transformed into the first image and the second image, the third digital filter group and the fourth digital filter group corresponding to a transform characteristic by which the light entering the single image-taking optical system is transformed into the third image and the fourth image,
wherein, for each pixel of the acquired first image and second image, based on the parallax for the pixel acquired by the parallax acquisition device, the filtering processing device reads the first transform filter and the second transform filter corresponding to the parallax, from the stored first transform filter group and second transform filter group, and then performs the filtering process using the read first transform filter and second transform filter.

7. The image processing device according to claim 6 , wherein the transform filter storage device stores a transform filter to be convoluted in real space, the transform filter being calculated by an amplitude component of the following formula,
F −1 (T f (ω x , ω y ) −1 ·T wf (ω x , ω y ))

(here, F −1 : Fourier inverse transform),
when any one digital filter of the first digital filter group and second digital filter group is T(x, y), the digital filter T(x, y) to which Fourier transform has been performed is T f (ω x , ω y ), a digital filter that is of the third digital filter group and fourth digital filter group and that corresponds to the digital filter T(x, y) is T w (x, y), and the digital filter T w (x, y) to which Fourier transform has been performed is T wf (ω x , ω y ).

8. The image processing device according to claim 6 , comprising a specification device for specifying a transform characteristic for the third image and the fourth image that are to be generated by the transform,
wherein the transform filter storage device stores the first transform filter group and the second transform filter group for each transform characteristic that are calculated based on a first digital filter group and a second digital filter group for each parallax and a third digital filter group and a fourth digital filter group for each parallax, the first digital filter group and the second digital filter group corresponding to a transform characteristic by which light entering the single image-taking optical system is transformed into the first image and the second image, the third digital filter group and the fourth digital filter group corresponding to a transform characteristic by which the light entering the single image-taking optical system is transformed into the third image and the fourth image and supporting multiple transform characteristics that can be specified by the specification device, and
the filtering processing device reads the first transform filter group and second transform filter gr

Assignees

• Fujifilm Corp

Inventors

• Izawa Katsutoshi
• Ono Shuji
• Hayashi Junji
• Kawai Tomoyuki
• Aoki Takashi

Classifications

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N13/0257

  Electricity · mapped topic

• H04N13/0018 Primary

  Electricity · mapped topic

• H04N13/0207

  Electricity · mapped topic