Image pickup apparatus, image processing apparatus, image processing method, and non-transitory computer-readable storage medium

Patent metadata

Field	Value
Publication number	US-2016005151-A1
Application number	US-201414652942-A
Country	US
Kind code	A1
Filing date	Jan 27, 2014
Priority date	Feb 1, 2013
Publication date	Jan 7, 2016
Grant date	—

Abstract

An image pickup apparatus includes an image pickup element which obtains an original image, a storage unit which stores a first optical transfer function different depending on an image pickup condition and on a position in the original image and an image restoration filter or a function designed to generate a second optical transfer function, a data obtaining unit which obtains the first optical transfer function when the aperture value is less than a predetermined value, and the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function when the aperture value is not less than the predetermined value, a filter generating unit which generates an image restoration filter based on the first or second optical transfer function, and an image processing unit which generates the restored image by using the image restoration filter.

First claim

1 . An image pickup apparatus capable of generating a restored image from an original image, the image pickup apparatus comprising:
an image pickup element configured to photoelectrically convert an object image formed via an image pickup optical system to obtain the original image; a storage unit configured to store a first optical transfer function which is different depending on an image pickup condition and on a position in the original image, and an image restoration filter whose distribution of two-dimensional filter values is constant with respect to a position in the original image and different depending on an aperture value or a function designed to generate a second optical transfer function; a data obtaining unit configured to obtain the first optical transfer function when the aperture value is less than a predetermined value, and obtain the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function when the aperture value is not less than the predetermined value; a filter generating unit configured to generate an image restoration filter based on the first optical transfer function or the second optical transfer function when the data obtaining unit obtains the first optical transfer function or the second optical transfer function; and an image processing unit configured to generate the restored image from the original image by using the image restoration filter stored in the storage unit or the image restoration filter generated by the filter generating unit.

2 . The image pickup apparatus according to claim 1 , wherein the function designed to generate the second optical transfer function stored in the storage unit is a calculation expression of an optical transfer function which depends on the aperture value.

3 . The image pickup apparatus according to claim 1 , wherein the image restoration filter is generated based further on a third optical transfer function depending on an optical low-pass filter.

4 . The image pickup apparatus according to claim 3 , wherein the image restoration filter is generated based further on a fourth optical transfer function depending on a pixel aperture.

5 . The image pickup apparatus according to claim 1 , wherein the image restoration filter satisfies 1<T·p/(0.0016·F)<6 where T is a tap number of the image restoration filter, p is a pixel pitch of the image pickup element, and F is the aperture value.

6 . The image pickup apparatus according to claim 1 , wherein the predetermined value of the aperture value satisfies F=p/λ where F is the aperture value, p is a pixel pitch of the image pickup element, and λ is a predetermined wavelength of a wavelength band of a visible light.

7 . The image pickup apparatus according to claim 1 , wherein the image pickup condition includes an object distance and a focal length.

8 . An image processing apparatus capable of generating a restored image from an original image, the image processing apparatus comprising:
an obtaining unit configured to obtain the original image and an image pickup condition of the original image; a storage unit configured to store a first optical transfer function which is different depending on the image pickup condition and on a position in the original image, and an image restoration filter whose distribution of two-dimensional filter values is constant with respect to a position in the original image and different depending on an aperture value or a function designed to generate a second optical transfer function; a data obtaining unit configured to obtain the first optical transfer function when the aperture value is less than a predetermined value, and obtain the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function when the aperture value is not less than the predetermined value; a filter generating unit configured to generate an image restoration filter based on the first optical transfer function or the second optical transfer function when the data obtaining unit obtains the first optical transfer function or the second optical transfer function; and an image processing unit configured to generate the restored image from the original image by using the image restoration filter stored in the storage unit or the image restoration filter generated by the filter generating unit.

9 . An image processing method of generating a restored image from an original image, the method comprising the steps of:
obtaining the original image and an image pickup condition of the original image; obtaining a first optical transfer function from a storage unit, the storage unit storing the first optical transfer function which is different depending on the image pickup condition and on a position in the original image to be used when an aperture value of the image pickup condition is less than a predetermined value, and the storage unit storing an image restoration filter whose distribution of two-dimensional filter values is constant with respect to a position in the original image and different depending on the aperture value or a function designed to generate a second optical transfer function to be used when the aperture value is not less than the predetermined value; obtaining, from the storage unit, the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function when the aperture value is not less than the predetermined value; generating an image restoration filter based on the first optical transfer function or the second optical transfer function when a data obtaining unit obtains the first optical transfer function or the second optical transfer function; and generating the restored image from the original image by using the image restoration filter stored in the storage unit or the image restoration filter generated by a filter generating unit.

10 . A non-transitory computer-readable storage medium storing an image processing program to generate a restored image from an original image, the image processing program being configured to cause an information processing apparatus to execute the steps of:
obtaining the original image and an image pickup condition of the original image; obtaining a first optical transfer function from a storage unit, the storage unit storing the first optical transfer function which is different depending on the image pickup condition and on a position in the original image to be used when an aperture value of the image pickup condition is less than a predetermined value, and the storage unit storing an image restoration filter whose distribution of two-dimensional filter values is constant with respect to a position in the original image and different depending on the aperture value or a function designed to generate a second optical transfer function to be used when the aperture value is not less than the predetermined value; obtaining, from the storage unit, the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function if the aperture value is equal to or more than a predetermined value; generating an image restoration filter based on the first optical transfer function or the second optical transfer function when a data obtaining unit obtains the first optical transfer function or the second optical transfer function; and generating the restored image from the original image by using the image restoration filter stored in the storage unit or the image restoration filter generated by a fil

Assignees

• Canon Kk

Inventors

• Hatakeyama Koshi

Classifications

• G06T5/20 Primary

  using local operators · CPC title

• H04N25/615

  involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF] · CPC title

• H04N5/357

  Electricity · mapped topic

• G06T5/001 Primary

  Physics · mapped topic

• G06T7/004

  Physics · mapped topic