Image processing device and image processing method

Patent metadata

Field	Value
Publication number	US-9672605-B2
Application number	US-201314423818-A
Country	US
Kind code	B2
Filing date	Sep 26, 2013
Priority date	Sep 27, 2012
Publication date	Jun 6, 2017
Grant date	Jun 6, 2017

Abstract

To increase the processing speed when median processing in the time direction is performed on color images, pixel values of each channel (R, G and B values) are extracted from multiple temporally consecutive frame images; subsequently, identifiers (index numbers) corresponding to frame images from which the pixel values are extracted are assigned to the pixel values; subsequently, a sorting process is performed on a set of pixel values of one channel (G values) to obtain an identifier assigned to a median value thereof; subsequently, a pixel value to which an identifier identical with that assigned to the median value is assigned is selected as a median value of each channel; and subsequently, a median image is generated on the basis of the median value of each channel obtained for each pixel.

First claim

The invention claimed is:

1. An image processing device for generating a snowfall-corrected image which has snowfall noise removed from temporally consecutive input images, the device comprising:
a processor that executes instructions, the instructions, when executed by the processor, causes the processor to perform operations including:
performing median processing in a time direction on temporally consecutive input images, the input images including an image at a time point of interest;
generating a median image at the time point of interest by performing the median processing;
generating a primary image at the time point of interest based on the input image at the time point of interest, the median image at the time point of interest, and a difference between a median image at a time point immediately preceding the time point of interest and the input image at the time point of interest;
generating a background image at the time point of interest based on the primary image at the time point of interest and a background image at the time point immediately preceding the time point of interest;
determining whether each pixel of the input image at the time point of interest represents an object in motion other than snow particle based on the input image at the time point of interest, an input image at a time point immediately subsequent to the time point of interest, and the background image at the time point of interest; and
generating the snowfall-corrected image:
when a pixel value of the input image at the time point of interest is adopted as a pixel value of the snowfall-corrected image when a pixel of the input image at the time point of interest represents the object in motion, and
when a pixel value of the median image at the time point of interest is adopted as the pixel value of the snowfall-corrected image when the pixel of the input image at the time point of interest represents the snow particle.

2. The image processing device according to claim 1 , further comprising a display device that displays the snowfall-corrected image.

3. The image processing device according to claim 1 , wherein the input images are images captured by an imaging device.

4. The image processing device according to claim 1 , wherein the processor further performs:
calculating an amount of change in brightness between a brightness of the input image at the time point of interest and a brightness of the median image or the input image at the time point immediately preceding the time point of interest; and
determining a significant change has appeared in the object in motion when the amount of change in brightness is greater than a first predetermined threshold value.

5. The image processing device according to claim 4 , wherein the processor further performs:
determining the pixel value of the median image at the time point of interest as a pixel value of the primary image at the time point of interest when the amount of change in brightness is less than or equal to the first threshold value.

6. The image processing device according to claim 4 , wherein the processor further performs:
determining the pixel value of the input image at the time point of interest as a pixel value of the primary image at the time point of interest when the amount of change in brightness is greater than the first predetermined threshold value.

7. The image processing device according to claim 1 , wherein the processor further performs:
determining, in an initialization process, the pixel value of the input image at the time point of interest as a pixel value of the median image at the time point immediately preceding the time point of interest and a pixel value of the background image at the time point immediately preceding the time point of interest.

8. The image processing device according to claim 4 , wherein the first threshold value is a positive value.

9. The image processing device according to claim 1 , wherein the processor further performs:
determining that,
the pixel value of the input image at the time point of the interest represents the object in motion when a pixel value, corresponding to the pixel value of the input image at the time point of the interest, successively deviates from a second threshold value over a predetermined number of the temporally consecutive input images, and
the pixel value of the input image at the time point of the interest represents the snow particle, a background, or a substantially stationary object when the pixel value, corresponding to the pixel value of the input image at the time point of the interest, does not successively deviate from second threshold value over the predetermined number of the temporally consecutive input images.

10. The image processing device according to claim 1 , wherein the temporally consecutive input images further include the input image at the time point immediately subsequent to the time point of interest.

11. An image processing method for generating a snowfall-corrected image which has snowfall noise removed from temporally consecutive input images, the method comprising:
performing median processing in a time direction on temporally consecutive input images, the input images including an image at a time point of interest;
generating median image at the time point of interest by performing the median processing;
generating a primary image at the time point of interest based on the input image at the time point of interest, the median image at the time point of interest, and the difference between a median image at a time point immediately preceding the time point of interest and the input image at the time point of interest;
generating a background image at the time point of interest based on the primary image at the time point of interest and a background image at the time point immediately preceding the time point of interest;
determining whether each pixel of the input image at the time point of interest represents an object in motion other than snow particle based on the input image at the time point of interest, an input image at a time point immediately subsequent to the time point of interest and the background image at the time point of interest; and
generating the snowfall-corrected image:
when a pixel value of the input image at the time point of interest is adopted as a pixel value of the snowfall-corrected image when a pixel of the input image at the time point of interest represents the object in motion, and
when a pixel value of the median image at the time point of interest is adopted as the pixel value of the snowfall-corrected image when the pixel of the input image at the time point of interest represents the snow particle.

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Matsuyama Yoshiyuki
• Tabei Kenji

Classifications

• G06T2207/30232

  Surveillance · CPC title

• G06T2207/20032

  Median filtering · CPC title

• H04N9/646

  for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/20216

  Image averaging · CPC title