Image processing device and method, and image capturing device

Patent metadata

Field	Value
Publication number	US-9288472-B2
Application number	US-201414534343-A
Country	US
Kind code	B2
Filing date	Nov 6, 2014
Priority date	May 9, 2012
Publication date	Mar 15, 2016
Grant date	Mar 15, 2016

Abstract

An image processing device comprising: an image acquisition device; a parallax acquisition device; a first data transform device; an operation processing device; and a second data transform device transforming third frequency component data and fourth frequency component data respectively corresponding to the third image and the fourth image calculated by the operation processing device into data on a real space and for respectively selecting pixels at positions corresponding to the target pixels as one pixel of the third image and one pixel of the fourth image.

First claim

What is claimed is:

1. An image processing device comprising:
an image acquisition device acquiring a first image and a second image that are captured through a single image-taking optical system and that have a parallax to each other after a pupil division by a pupil-division device;
a parallax acquisition device acquiring a parallax between corresponding pixels of the acquired first image and second image;
a first data transform device transforming, for each of a plurality of target pixels of processing of image data of the acquired first image and image data of the acquired second image, predetermined regions based on the target pixel into first frequency component data and second frequency component data, respectively;
an operation processing device performing a multiplication process for each of the frequency component data of the acquired first image and second image by using a first frequency filter group and a second frequency filter group for respectively transforming the transformed first frequency component data and second frequency component data in a frequency space and for respectively transforming the acquired first image and second image to a third image and a fourth image in which a parallax amount and a blur amount of the acquired first image and second image have been altered, the first frequency filter and the second frequency filter corresponding to the parallax of the target pixels acquired by the parallax acquisition device; and
a second data transform device transforming third frequency component data and fourth frequency component data respectively corresponding to the third image and the fourth image calculated by the operation processing device into data on a real space and for respectively selecting pixels at positions corresponding to the target pixels as one pixel of the third image and one pixel of the fourth image.

2. The image processing device according to claim 1 , wherein
the operation processing device uses a first frequency filter and a second frequency filter that increase the blur amount with an increase in the parallax of the target pixels acquired by the parallax acquisition device.

3. The image processing device according to claim 1 , further comprising:
a digital filter storage device storing a first digital filter group and a second digital filter group for each parallax respectively corresponding to transform characteristics by which light entering the single image-taking optical system is transformed into the first image and the second image; and
a frequency filter calculation device calculating the first frequency filter group and the second frequency filter group based on the stored first digital filter group and second digital filter group and based on a third digital filter group and a fourth digital filter group for each parallax respectively corresponding to transform characteristics by which light entering the single image-taking optical system is transformed into the third image and the fourth image, wherein
for each pixel of the acquired first image and second image, the operation processing device selects the first frequency filter and the second frequency filter corresponding to the parallax from the calculated first frequency filter group and second frequency filter group, respectively, based on the parallax of the pixels acquired by the parallax acquisition device and uses the selected first frequency filter and second frequency filter to perform the multiplication process.

4. The image processing device according to claim 3 , wherein
the frequency filter calculation device calculates the first and second frequency filters G(ωx, ωy) by using the following formula
G (ω x,ωy )= Twf (ω x,ωy )· Tf (ω x,ωy )−1,

wherein any one digital filter of the first digital filter group and the second digital filter group stored in the digital filter storage device is T(x, y), the digital filter T(x, y) to which Fourier transform has been performed is Tf(ωx, ωy), a digital filter corresponding to the digital filter T(x, y) of the third digital filter group and the fourth digital filter group is Tw(x, y), and the digital filter Tw(x, y) to which Fourier transform has been performed is Twf(ωx, ωy).

5. The image processing device according to claim 3 , further comprising:
a designation device designating the transform characteristics for the transform into the third image and the fourth image; and
a digital filter calculation device calculating the third digital filter group and the fourth digital filter group corresponding to the designated transform characteristics, wherein
the frequency filter calculation device uses the first digital filter group and the second digital filter group stored in the digital filter storage device and the third digital filter group and the fourth digital filter group calculated by the digital filter calculation device to calculate the first frequency filter group and the second frequency filter group.

6. The image processing device according to claim 1 , further comprising
a frequency filter storage device storing the first frequency filter group and the second frequency filter group for each parallax calculated based on the first digital filter group and the second digital filter group for each parallax respectively corresponding to the transform characteristics by which the light entering the single image-taking optical system is transformed into the first image and the second image and based on the third digital filter group and the fourth digital filter group for each parallax respectively corresponding to the transform characteristics by which the light entering the single image-taking optical system is transformed into the third image and the fourth image, wherein
for each pixel of the acquired first image and second image, the operation processing device reads the first frequency filter and the second frequency filter corresponding to the parallax from the stored first frequency filter group and second frequency filter group based on the parallax of the pixels acquired by the parallax acquisition device and uses the read first frequency filter and second frequency filter to perform the multiplication process.

7. The image processing device according to claim 6 , wherein
the frequency filter storage device stores the first and second frequency filters G(ωx, ωy) calculated by the following formula
G (ω x,ωy )= Twf (ω x,ωy )· Tf (ω x,ωy )−1,

wherein any one digital filter of the first digital filter group and the second digital filter group is T(x, y), the digital filter T(x, y) to which Fourier transform has been performed is Tf(ωx, ωy), the digital filter corresponding to the digital filter T(x, y) of the third digital filter group and the fourth digital filter group is Tw(x, y), and the digital filter Tw(x, y) to which Fourier transform has been performed is Twf(ωx, ωy).

8. The image processing device according to claim 6 , further comprising
a designation device designating the transform characteristics for the transform into the third image and the fourth image, wherein
the frequency filter storage device stores the first frequency filter group and the second frequency filter group for each transform characteristic calculated based on the first digital filter group and the second digital filter group for each parallax corresponding to the transform characteristics by which the light entering the single image-taking optical system is transformed into the first image and the second image and based on the third digital filter group and the fourth digital filter group for each parallax corresponding to the transform characteristics by which the light entering the single image-taking optical system is transformed into the third image and the fourth image,

Assignees

• Fujifilm Corp

Inventors

• Aoki Takashi
• Izawa Katsutoshi

Classifications

• H04N13/0217

  Electricity · mapped topic

• H04N13/0225 Primary

  Electricity · mapped topic

• H04N13/218 Primary

  using spatial multiplexing · CPC title

• H04N13/225 Primary

  using parallax barriers · CPC title