Restoration filter generation device and method, image processing device and method, imaging device, and non-transitory computer-readable medium

Patent metadata

Field	Value
Publication number	US-9727955-B2
Application number	US-201514837779-A
Country	US
Kind code	B2
Filing date	Aug 27, 2015
Priority date	Mar 4, 2013
Publication date	Aug 8, 2017
Grant date	Aug 8, 2017

Abstract

A restoration filter generation device which generates a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the restoration filter generation device including an MTF acquisition device which acquires a modulation transfer function MTF for the optical system; and a restoration filter generation device which generates the restoration filter based on the modulation transfer function MTF, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a region of a particular spatial frequency or less, the image data for each color of the multiple colors corresponding to the luminance system image data after the restoration process.

First claim

What is claimed is:

1. A restoration filter generation device which generates a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the image data for each color of the multiple colors being obtained by an imaging device including the optical system,
the restoration filter generation device comprising:
a modulation transfer function MTF acquisition device which acquires a modulation transfer function MTF for the optical system;
a restoration filter generation unit which generates the restoration filter based on the modulation transfer function MTF acquired by the MTF acquisition device, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a region of a particular spatial frequency or less, the image data for each color of the multiple colors corresponding to the luminance system image data after the restoration process; and
an amplification factor acquisition device which acquires an amplification factor of the MTF value for each spatial frequency with respect to image data for each color of the multiple colors, the amplification factor corresponding to an amplification factor of the MTF value for each spatial frequency with respect to the luminance system image data after the restoration process, the image data for each color of the multiple colors being generated by an inverse conversion process of the luminance system image data,
wherein: the MTF acquisition device acquires the modulation transfer function MTF for each color of the multiple colors, and
the restoration filter generation unit determines frequency characteristics of restoration filters for the respective colors, based on the modulation transfer functions MTF for the respective colors respectively, selects a minimum value of the amplification factors of the restoration filters for the respective colors, for each spatial frequency, based on the frequency characteristics, and generates the restoration filter based on the minimum value of the amplification factor for each spatial frequency and an acquisition result by the amplification factor acquisition device.

2. The restoration filter generation device according to claim 1 ,
wherein the particular spatial frequency is equal to or less than a half of a Nyquist frequency of an imaging element included in the imaging device.

3. The restoration filter generation device according to claim 1 ,
wherein the MTF acquisition device acquires the modulation transfer function MTF for the optical system including a lens unit that modulates a phase and extends a depth of field.

4. An image processing device comprising:
an image data generation device which generates luminance system image data based on image data for each color of multiple colors, the luminance system image data being image data relevant to luminance, the image data for each color of the multiple colors being obtained by an imaging device including an optical system;
a restoration filter storage device which stores the restoration filter generated by the restoration filter generation device according to claim 1 ; and
a restoration processing device which performs a restoration process on the luminance system image data generated by the image data generation device, using the restoration filter stored in the restoration filter storage device.

5. An imaging system comprising:
an imaging device which outputs image data for each color of multiple colors, the imaging device including an optical system; and
the image processing device according to claim 4 .

6. A restoration filter generation method for generating a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the image data for each color of the multiple colors being obtained by an imaging device including the optical system,
the restoration filter generation method comprising:
a modulation transfer function MTF acquisition step of acquiring a modulation transfer function MTF for the optical system;
a restoration filter generation step of generating the restoration filter based on the modulation transfer function MTF acquired in the MTF acquisition step, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a region of a particular spatial frequency or less, the image data for each color of the multiple colors corresponding to the luminance system image data after the restoration process; and
an amplification factor acquisition step of acquiring an amplification factor of the MTF value for each spatial frequency with respect to image data for each color of the multiple colors, the amplification factor corresponding to an amplification factor of the MTF value for each spatial frequency with respect to the luminance system image data after the restoration process, the image data for each color of the multiple colors being generated by an inverse conversion process of the luminance system image data,
wherein: the MTF acquisition step acquires the modulation transfer function MTF for each color of the multiple colors, and
the restoration filter generation step determines frequency characteristics of restoration filters for the respective colors, based on the modulation transfer functions MTF for the respective colors respectively, selects a minimum value of the amplification factors of the restoration filters for the respective colors, for each spatial frequency, based on the frequency characteristics, and generates the restoration filter based on the minimum value of the amplification factor for each spatial frequency and an acquisition result by the amplification factor acquisition step.

7. An image processing method comprising:
an image data generation step of generating luminance system image data based on image data for each color of multiple colors, the luminance system image data being image data relevant to luminance, the image data for each color of the multiple colors being obtained by an imaging device including an optical system; and
a restoration processing step of performing a restoration process on the luminance system image data generated in the image data generation step, using the restoration filter generated by the restoration filter generation method according to claim 6 .

8. A non-transitory computer-readable medium recording a program for generating a restoration filter for performing a restoration process on luminance system image data, the restoration process being based on a point-image distribution in an optical system, the luminance system image data being image data relevant to luminance and being generated based on image data for each color of multiple colors, the image data for each color of the multiple colors being obtained by an imaging device including the optical system,
the program causing a computer to execute:
a modulation transfer function MTF acquisition step of acquiring a modulation transfer function MTF for the optical system;
a restoration filter generation step of generating the restoration filter based on the modulation transfer function MTF acquired in the MTF acquisition step, the restoration filter suppressing an MTF value of image data for each color of the multiple colors to 1.0 or less at least in a

Assignees

• Fujifilm Corp

Inventors

• Naruse Yousuke
• Hayashi Kenkichi
• Sugimoto Masahiko

Classifications

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N25/615

  involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF] · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N25/6153

  for colour signals · CPC title

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title