Image processing device, imaging device, image processing method, and program

Patent metadata

Field	Value
Publication number	US-10326947-B2
Application number	US-201715789146-A
Country	US
Kind code	B2
Filing date	Oct 20, 2017
Priority date	Apr 23, 2015
Publication date	Jun 18, 2019
Grant date	Jun 18, 2019

Abstract

The present invention provides an image processing device, an imaging device, an image processing method, and a program which are capable of accurately correcting blurring caused in first image data of an image using a near-infrared ray as a light source and, accurately performing a point image restoration process on second image data of an image using visible light and a near-infrared ray as a light source. An image processing device according to an aspect of the present invention includes an image input unit, a determination unit that determines whether image data is first image data or second image data, a first restoration processing unit that performs a first restoration process using first restoration filters for performing phase correction and amplitude restoration on the determined first image data, and a second restoration processing unit that performs a second restoration process using second restoration filters for performing amplitude restoration without phase correction on the determined second image data.

First claim

What is claimed is:

1. An image processing device comprising:
an image input unit that receives image data imaged with sensitivity to a visible light wavelength band and a near-infrared ray wavelength band by using an optical system;
a determination unit that determines whether the image data is first image data using a near-infrared ray as a light source or second image data using a near-infrared ray and visible light as a light source;
a first restoration processing unit that performs a first restoration process using first restoration filters for performing phase correction and amplitude restoration on the first image data determined by the determination unit, the first restoration filters being based on a first point spread function for a near-infrared ray of the optical system; and
a second restoration processing unit that performs a second restoration process using second restoration filters for performing amplitude restoration without phase correction on the second image data determined by the determination unit, the second restoration filters being based on a second point spread function for visible light and a near-infrared ray of the optical system.

2. The image processing device according to claim 1 , further comprising a light amount detection unit that detects a light amount of the image data,
wherein the determination unit determines whether the image data is the first image data or the second image data based on the light amount detected by the light amount detection unit.

3. The image processing device according to claim 1 , wherein the determination unit determines whether the image data is the first image data or the second image data based on a time when the image data is acquired.

4. The image processing device according to claim 1 , further comprising a tone correction processing unit that performs non-linear tone correction on the first image data, wherein:
the tone correction processing unit performs the non-linear tone correction on the first image data on which the phase correction is performed; and
the first restoration processing unit performs the amplitude restoration on the first image data on which the non-linear tone correction is performed.

5. The image processing device according to claim 1 , further comprising a tone correction processing unit that performs non-linear tone correction on the first image data, wherein:
the tone correction processing unit performs the non-linear tone correction on the first image data on which the amplitude restoration is performed; and
the first restoration processing unit performs the phase correction on the first image data on which the non-linear tone correction is performed.

6. The image processing device according to claim 1 , further comprising at least one of a common restoration process arithmetic unit that is used in a restoration process arithmetic of the first restoration processing unit and the second restoration processing unit, a common tone correction arithmetic unit that performs non-linear tone correction on the first image data and the second image data, or a common contour emphasis processing unit that performs a contour emphasis process on the first image data and the second image data.

7. The image processing device according to claim 1 , further comprising a light amount ratio detection unit that detects a light amount ratio between a light amount of the visible light and a light amount of the near-infrared ray in the second image data,
wherein the second restoration processing unit uses the second restoration filters generated based on a modulation transfer function of the visible light of the optical system and a modulation transfer function of the near-infrared ray of the optical system depending on the light amount ratio detected by the light amount ratio detection unit.

8. The image processing device according to claim 1 , further comprising a storage unit that stores the first restoration filters and the second restoration filters.

9. The image processing device according to claim 1 , further comprising a filter generation unit that generates the first restoration filters and the second restoration filters.

10. An imaging device comprising:
an optical system;
a near-infrared ray emitting unit that emits a near-infrared ray as auxiliary light;
an image acquisition unit that acquires image data imaged with sensitivity to a visible light wavelength band and a near-infrared ray wavelength band by using the optical system;
a determination unit that determines whether the image data is first image data using a near-infrared ray as a light source and second image data using a near-infrared ray and visible light as a light source;
a first restoration processing unit that performs a first restoration process using first restoration filters for performing phase correction and amplitude restoration on the acquired first image data, the first restoration filters being based on a first point spread function for a near-infrared ray of the optical system; and
a second restoration processing unit that performs a second restoration process using second restoration filters for performing amplitude restoration without phase correction on the acquired second image data, the second restoration filters being based on a second point spread function for visible light and a near-infrared ray of the optical system.

11. The imaging device according to claim 10 , wherein an image surface position is set using a case where the image acquisition unit images a visible light image as a criterion.

12. An image processing method comprising:
an image input step of receiving image data imaged with sensitivity to a visible light wavelength band and a near-infrared ray wavelength band by using an optical system;
a determination step of determining whether the image data is first image data using a near-infrared ray as a light source or second image data using a near-infrared ray and visible light as a light source;
a first restoration processing step of performing a first restoration process using first restoration filters for performing phase correction and amplitude restoration on the first image data determined in the determination step, the first restoration filters being based on a first point spread function for a near-infrared ray of the optical system; and
a second restoration processing step of performing a second restoration process using second restoration filters for performing amplitude restoration without phase correction on the second image data determined in the determination step, the second restoration filters being based on a second point spread function for visible light and a near-infrared ray of the optical system.

13. A non-transitory computer-readable tangible medium containing a program causing a computer to perform:
an image input step of receiving image data imaged with sensitivity to a visible light wavelength band and a near-infrared ray wavelength band by using an optical system;
a determination step of determining whether the image data is first image data using a near-infrared ray as a light source or second image data using a near-infrared ray and visible light as a light source;
a first restoration processing step of performing a first restoration process using first restoration filters for performing phase correction and amplitude restoration on the first image data determined in the determination step, the first restoration filters being based on a first point spread function for a near-infrared ray of the optical system; and
a second restoration processing step of performing a second restoration process using second restoration filters for perfor

Assignees

• Fujifilm Corp

Inventors

• Hayashi Kenkichi
• Tanaka Junichi
• Naruse Yousuke
• Sugimoto Masahiko

Classifications

• H04N9/01

  Circuitry for demodulating colour component signals modulated spatially by colour striped filters by phase separation · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N25/61

  the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" · CPC title

• H04N25/6153

  for colour signals · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title