Restoration filter generation device and method, image processing device and method, imaging device, and non-transitory computer-readable medium

Patent metadata

Field	Value
Publication number	US-9984448-B2
Application number	US-201715632641-A
Country	US
Kind code	B2
Filing date	Jun 26, 2017
Priority date	Mar 4, 2013
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

A restoration filter generation device which generates a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the restoration filter generation device including an MTF acquisition device which acquires a modulation transfer function MTF for the optical system; and a restoration filter generation device which generates the restoration filter based on the modulation transfer function MTF, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a region of a particular spatial frequency or less, the image data for each color of the multiple colors corresponding to the luminance system image data after the restoration process.

First claim

What is claimed is:

1. A restoration filter generation device which generates a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the image data for each color of the multiple colors being obtained by an imaging device including the optical system, the restoration filter generation device comprising:
a modulation transfer function MTF acquisition device which acquires a modulation transfer function MTF for the optical system; and
a restoration filter generation unit which generates the restoration filter based on the modulation transfer function MTF acquired by the MTF acquisition device, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a region of a particular spatial frequency or less, the image data for each color of the multiple colors corresponding to the luminance system image data after the restoration process, wherein:
the MTF acquisition device acquires the modulation transfer function MTF for each color of the multiple colors; and
the restoration filter generation unit selects a maximum value of the MTF value from the modulation transfer functions MTF for the respective colors, for each spatial frequency, determines a frequency characteristic of a Wiener filter, based on the maximum value of the MTF value for each spatial frequency, and generates the restoration filter that achieves the frequency characteristic in a luminance system.

2. The restoration filter generation device according to claim 1 , wherein the particular spatial frequency is equal to or less than a half of a Nyquist frequency of an imaging element included in the imaging device.

3. The restoration filter generation device according to claim 1 , wherein the MTF acquisition device acquires the modulation transfer function MTF for the optical system including a lens unit that modulates a phase and extends a depth of field.

4. An image processing device comprising:
an image data generation device which generates luminance system image data based on image data for each color of multiple colors, the luminance system image data being image data relevant to luminance, the image data for each color of the multiple colors being obtained by an imaging device including an optical system;
a restoration filter storage device which stores the restoration filter generated by the restoration filter generation device according to claim 1 ; and
a restoration processing device which performs a restoration process on the luminance system image data generated by the image data generation device, using the restoration filter stored in the restoration filter storage device.

5. An imaging system comprising:
an imaging device which outputs image data for each color of multiple colors, the imaging device including an optical system; and
the image processing device according to claim 4 .

6. A restoration filter generation method for generating a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the image data for each color of the multiple colors being obtained by an imaging device including the optical system, the restoration filter generation method comprising:
a modulation transfer function MTF acquisition step of acquiring a modulation transfer function MTF for the optical system; and
a restoration filter generation step of generating the restoration filter based on the modulation transfer function MTF acquired in the MTF acquisition step, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a region of a particular spatial frequency or less, the image data for each color of the multiple colors corresponding to the luminance system image data after the restoration process, wherein:
the MTF acquisition step acquires the modulation transfer function MTF for each color of the multiple colors; and
the restoration filter generation step selects a maximum value of the MTF value from the modulation transfer functions MTF for the respective colors, for each spatial frequency, determines a frequency characteristic of a Wiener filter, based on the maximum value of the MTF value for each spatial frequency, and generates the restoration filter that achieves the frequency characteristic in a luminance system.

7. An image processing method comprising:
an image data generation step of generating luminance system image data based on image data for each color of multiple colors, the luminance system image data being image data relevant to luminance, the image data for each color of the multiple colors being obtained by an imaging device including an optical system; and
a restoration processing step of performing a restoration process on the luminance system image data generated in the image data generation step, using the restoration filter generated by the restoration filter generation method according to claim 6 .

8. A non-transitory computer-readable medium recording a program for generating a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the image data for each color of the multiple colors being obtained by an imaging device including the optical system, the program causing a computer to execute:
a modulation transfer function MTF acquisition step of acquiring a modulation transfer function MTF for the optical system; and
a restoration filter generation step of generating the restoration filter based on the modulation transfer function MTF acquired in the MTF acquisition step, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a region of a particular spatial frequency or less, the image data for each color of the multiple colors corresponding to the luminance system image data after the restoration process, wherein:
the MTF acquisition step acquires the modulation transfer function MTF for each color of the multiple colors; and
the restoration filter generation step selects a maximum value of the MTF value from the modulation transfer functions MTF for the respective colors, for each spatial frequency, determines a frequency characteristic of a Wiener filter, based on the maximum value of the MTF value for each spatial frequency, and generates the restoration filter that achieves the frequency characteristic in a luminance system.

Assignees

• Fujifilm Corp

Inventors

• Naruse Yousuke
• Hayashi Kenkichi
• Sugimoto Masahiko

Classifications

• H04N25/615

  involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF] · CPC title

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N25/6153

  for colour signals · CPC title