Image fusion method and apparatus using multi-spectral filter array sensor

Patent metadata

Field	Value
Publication number	US-9285309-B2
Application number	US-201313953895-A
Country	US
Kind code	B2
Filing date	Jul 30, 2013
Priority date	Feb 4, 2013
Publication date	Mar 15, 2016
Grant date	Mar 15, 2016

Abstract

Provided is an image fusion apparatus using a multi-spectral filter array sensor, the image fusion apparatus including a separation unit configured to separate an input image input through the multi-spectral filter array sensor into an RGB image and a near infrared (NIR) image, a color reconstruction unit configured to extract an RGB image in a visible band by removing an NIR value incident together with R, G and B values through the multi-spectral filter array sensor from the R, G and B values, and an image fusion unit configured to fuse the RGB image in the visible band with the NIR image, in which a fusion rate of the RGB image and the NIR image is based on a size of each of the R, G and B values of the RGB image in the visible band.

First claim

What is claimed is:

1. An image fusion apparatus using a multi-spectral filter array sensor, the image fusion apparatus comprising at least one processor to implement:
a separation unit configured to separate an input image input through the multi-spectral filter array sensor into an RGB image and a near infrared (NIR) image;
a color reconstruction unit configured to extract an RGB image in a visible band by removing an NIR value incident together with R, G and B values through the multi-spectral filter array sensor from the R, G and B values; and
an image fusion unit configured to fuse the RGB image in the visible band with the NIR image, in which a fusion rate of the RGB image and the NIR image is based on a size of each of the R, G and B values of the RGB image in the visible band,
wherein respective color channels R′, G′, and B′ fused by the image fusion unit are given by:

(

R
′

G
′

B
′

)

=

(

1
+
k

0

0

0

1
+
k

0

0

0

1
+
k

)

·

(

R

G

B

)

,

where

k
=

3
⁢
N

R
+
G
+
B

,

and N indicates the NIR image.

2. The image fusion apparatus of claim 1 , wherein the multi-spectral filter array sensor comprises a color filter which passes light components of the visible band ranging from 400 nm to 700 nm, such as RGB, and an NIR filter which passes light components of an NIR zone of 700 nm to 1100 nm in the invisible band.

3. The image fusion apparatus of claim 1 , wherein the color reconstruction unit extracts the RGB image in the visible band by removing the NIR value from each of all band values of color channels comprising R, G and B channels of the RGB image.

4. The image fusion apparatus of claim 1 , wherein a brightness of an image resulting from fusion by the image fusion unit is I′ (Intensity)=(R′+G′+B′)/3.

5. An image fusion apparatus using a multi-spectral filter array sensor, the image fusion apparatus comprising at least one processor to implement:
a separation unit configured to separate an input image input through the multi-spectral filter array sensor into an RGB image and a near infrared (NIR) image;
a color reconstruction unit configured to extract an RGB image in a visible band by removing an NIR value incident together with R, G and B values through the multi-spectral filter array sensor from the R, G and B values; and
an image fusion unit configured to fuse the RGB image in the visible band with the NIR image, in which a fusion rate of the RGB image and the NIR image is based on a size of each of the R, G and B values of the RGB image in the visible band,
wherein the image fusion unit is further configured to adjust the fusion rate according to local or global characteristics of the input image, and
wherein respective color channels R′, G′, and B′ fused by the image fusion unit are given by:
R′=N ×(1−α)× R /( R+G+B )+ N×α

G′=N ×(1−α)× G /( R+G+B )+ N×α

B′=N ×(1−α)× B /( R+G+B )+ N×α,

where α is implemented to be changed according to the local or global characteristics of the input image.

6. The image fusion apparatus of claim 5 , wherein the color reconstruction unit extracts the RGB image in the visible band by removing NIR values from each of all band values of color channels comprising R, G and B channels of the RGB image.

7. An image fusion method using a multi-spectral filter array sensor, the image fusion method comprising:
extracting an RGB image in a visible band from an RGB image input through the multi-spectral filter array sensor;
receiving input of an NIR image through an NIR filter of the multi-spectral filter array sensor;
calculating a size of each of R, G and B values of the RGB image in the visible band; and
fusing the RGB image in the visible band with the NIR image according to the calculated size of each of R, G, and B pixels,
wherein respective color channels R′, G′, and B′ fused in the fusing are given by:

(

R
′

G
′

B
′

Assignees

• Hanwha Techwin Co Ltd

Inventors

• Choi Eun-Cheol

Classifications

• H04N23/12

  with one sensor only · CPC title

• H04N23/11 Primary

  for generating image signals from visible and infrared light wavelengths · CPC title

• H04N9/07

  Electricity · mapped topic

• H04N5/332

  Electricity · mapped topic

• G01N21/359 Primary

  using near infrared light · CPC title