Imaging device, image processing method and program for imaging device

Patent metadata

Field	Value
Publication number	US-10634830-B2
Application number	US-201715789196-A
Country	US
Kind code	B2
Filing date	Oct 20, 2017
Priority date	Apr 23, 2015
Publication date	Apr 28, 2020
Grant date	Apr 28, 2020

Abstract

For causing a near-infrared light image obtained through imaging in a second imaging mode to be easily viewed while suppressing an increase in size and number of components, in a case where an imaging unit is operating in the second imaging mode, a change region where there is a change is detected through comparison with the second image data that the imaging unit has acquired before the second image data, from within the second image data acquired by the imaging unit. Combination image data obtained by combining the change region image data corresponding to the change region within the second image data with background image data of a color image generated on the basis of first image data acquired in a first imaging mode immediately before the second imaging mode by the imaging unit and corresponding to a background region different from the change region within the second image data is generated on the basis of the detection result of the change region.

First claim

What is claimed is:

1. An imaging device installed at a fixed position, comprising:
an optical system in which an infrared ray cut filter is insertable into an imaging optical path or retractable from the imaging optical path;
an imaging unit including signal processing circuitry and having a first imaging mode in which the imaging unit images a subject through the optical system in which the infrared ray cut filter has been inserted into the imaging optical path to acquire first image data indicating a visible light image that is a visible light image of the subject, the visible light image being a color image, and a second imaging mode in which the imaging unit images the subject through the optical system in which the infrared ray cut filter has been retracted from the imaging optical path to acquire second image data including a near-infrared light image that is a near-infrared light image of the subject, the near-infrared light image being a monochrome image; and
processing circuitry configured to:
alternately switch between the first imaging mode and the second imaging mode;
detect a change region having a change within the second image data acquired by the imaging unit when compared with the second image data that the imaging unit has acquired prior to the second image data in a case where the imaging unit is operating in the second imaging mode;
generate combination image data obtained by combining the change region image data corresponding to the change region within the second image data with background image data of the color image generated on the basis of the first image data acquired in the first imaging mode immediately before the second imaging mode by the imaging unit and corresponding to a background region different from the change region within the second image data on the basis of the detection result; and
perform a point image restoration process based on first optical characteristics of the optical system with respect to near-infrared light on at least the change region image data,
wherein the processing circuitry is further configured to generate the background image data on the basis of color information extracted from a region corresponding to the background region within the first image data, and the luminance information in the background region of the second image data,
wherein the first optical characteristics are a first point spread function of the optical system with respect to the near-infrared light; and
wherein the point image restoration process that the processing circuitry performs on the change region image data includes restoring an amplitude component of the change region image data, and a phase component due to a shape of the non-point-symmetric, first point spread function.

2. The imaging device according to claim 1 , wherein the processing circuitry is further configured to generate the background image data using the image data itself included in the region corresponding to the background region within the first image data.

3. The imaging device according to claim 1 , wherein the processing circuitry is further configured to detect a non-change region in the plurality of pieces of first image data that the imaging unit has acquired at different timings in the first imaging mode, and
generate the background image data from the image data corresponding to the non-change region within the first image data on the basis of a detection result.

4. The imaging device according to claim 2 , wherein the processing circuitry is further configured to detect a non-change region in the plurality of pieces of first image data that the imaging unit has acquired at different timings in the first imaging mode, and
generate the background image data from the image data corresponding to the non-change region within the first image data on the basis of a detection result.

5. The imaging device according to claim 1 , wherein the processing circuitry is further configured to perform a sharpening process on at least the change region image data.

6. The imaging device according to claim 1 , wherein the processing circuitry is further configured to perform a lateral chromatic aberration correction process on at least the change region image data.

7. The imaging device according to claim 1 , wherein the processing circuitry is further configured to perform a point image restoration process based on second optical characteristics of the optical system with respect to visible light on the first image data.

8. The imaging device according to claim 7 , wherein:
the second optical characteristics are a second point spread function of each color of the optical system with respect to red, green, and blue visible light; and
the point image restoration process that the processing circuitry performs on the first image data includes a process of restoring a phase component of the first image data using a restoration filter based on the second point spread function for each color.

9. An image processing method for an imaging device comprising an optical system in which an infrared ray cut filter is insertable into an imaging optical path or retractable from the imaging optical path, an imaging unit including signal processing circuitry and having a first imaging mode in which the imaging unit images a subject through the optical system in which the infrared ray cut filter has been inserted into the imaging optical path to acquire first image data indicating a visible light image that is a visible light image of the subject, the visible light image being a color image, and a second imaging mode in which the imaging unit images the subject through the optical system in which the infrared ray cut filter has been retracted from the imaging optical path to acquire second image data including a near-infrared light image that is a near-infrared light image of the subject, the near-infrared light image being a monochrome image, and processing circuitry that alternately switches between the first imaging mode and the second imaging mode, the imaging device being installed at a fixed position, the method comprising:
a change region detection step of detecting a change region having a change within the second image data acquired by the imaging unit when compared with the second image data that the imaging unit has acquired prior to the second image data in a case where the imaging unit is operating in the second imaging mode;
a combination image data generation step of generating combination image data obtained by combining the change region image data corresponding to the change region within the second image data with background image data of the color image generated on the basis of the first image data acquired in the first imaging mode immediately before the second imaging mode by the imaging unit and corresponding to a background region different from the change region within the second image data on the basis of the detection result in the change region detection step; and
a first point image restoration processing step of performing a point image restoration process based on first optical characteristics of the optical system with respect to near-infrared light on at least the change region image data,
wherein the first optical characteristics are a first point spread function of the optical system with respect to the near-infrared light; and
wherein the point image restoration process that the first point image restoration processing step performs on the change region image data includes restoring an amplitude component of the change region image data, and a phase component due to a shape of the non-point-symmetric, first point spread function.

10. A non-transitory computer-readable tangible medium conta

Assignees

• Fujifilm Corp

Inventors

• Kunugise Takashi
• Naruse Yousuke

Classifications

• G06T7/254

  involving subtraction of images · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/30232

  Surveillance · CPC title

• G03B11/00 Primary

  Filters or other obturators specially adapted for photographic purposes · CPC title