Image processing device, image processing method and recording medium for reducing noise in image

Patent metadata

Field	Value
Publication number	US-10200649-B2
Application number	US-201514768934-A
Country	US
Kind code	B2
Filing date	Feb 5, 2015
Priority date	Feb 7, 2014
Publication date	Feb 5, 2019
Grant date	Feb 5, 2019

Abstract

An image processing device comprising: a recording unit configured to record a previous frame image input before a target frame image that is to be a processing target or an output frame image from the previous frame image; an alignment unit configured to align the target frame image with the previous frame image or with the output frame image from the previous frame image; a correction unit configured to perform a temporal correction process to correct a pixel value of the target frame image by use of a pixel value of the previous frame image or a pixel value of the output frame image from the previous frame image that has been aligned by the alignment unit, with reference to the recording unit; and a generation unit configured to generate an output frame image from the target frame image by use of the target frame image corrected by the correction unit.

First claim

The invention claimed is:

1. An image processing device for receiving and processing an input frame image captured by an imaging device to generate an output frame image, comprising:
a processor; and
a memory storing instructions executable by the processor,
wherein the processor is configured to:
record, in a storage, a recorded frame image comprising one of:
a previous input frame image that was input before a current input frame image that is to be a processing target; or
a previous output frame image generated from the previous input frame image;

align the current input frame image with the recorded frame image;
calculate a weighted average using a correction filter, wherein the weighted average is calculated by directly weighting a pixel value of a target pixel of the current input frame image and directly weighting pixel values of corresponding pixels of at least one of a plurality of previous input frame images that are a predetermined number of frames before the current input frame image;
perform a temporal correction process to correct the target pixel by using the weighted average and a pixel value of a corresponding pixel of one previous output frame image generated from any one of the plurality of previous input frame images; and
generate a current output frame image from the current input frame image by use of the current input frame image corrected by the temporal correction process,
wherein in the correction filter, weights on the pixel values of the corresponding pixels of the previous input frame images are set to decrease toward older previous input frame images, and
wherein in the correction filter, the weights on the pixel values of the corresponding pixels of the plurality of previous input frame images of the predetermined number of frames are set so that a difference from the weight on the pixel value of the corresponding pixel of the previous input frame image that is one frame after a given previous input frame image or from the weight on the pixel value of the target pixel is smaller than a difference between the weight on the pixel value of the corresponding pixel of the previous input frame image that is the predetermined number of frames ahead of the current input frame image and the weight on the pixel value of the corresponding pixel of the previous input frame image that is one frame before the previous input frame image that is the predetermined number of frames ahead of the current input frame image.

2. The image processing device according to claim 1 , wherein the processor is further configured to perform a spatial correction process to correct a pixel value of a pixel of the current input frame image by use of pixel values of a plurality of pixels included in a region including the aforementioned pixel in the current input frame image.

3. The image processing device according to claim 1 , wherein the processor is further configured to perform the temporal correction process for the target pixel, using a coefficient according to a difference between a pixel value of the target pixel of the current input frame image and a pixel value of a corresponding pixel of the previous input frame image or the pixel value of the corresponding pixel of the previous output frame image from the previous frame image that has been aligned by the alignment.

4. The image processing device according to claim 1 , wherein the processor is further configured to store the current output frame image in the storage.

5. The image processing device according to claim 1 , wherein the processor is further configured to determine a clipping region from the current input frame image corrected by the temporal correction process based on a motion of the imaging device, and clips the clipping region, thereby generating the current output frame image.

6. The image processing device according to claim 5 , wherein the processor is further configured to exclude a pixel from the processing target, the pixel being included in at least a part of a region different from the clipping region of the current input frame image.

7. The image processing device according to claim 1 , wherein the processor is further configured, using another correction filter, to perform the temporal correction process for the target pixel, using the pixel value of the target pixel of the current input frame image and the pixel value of the corresponding pixel of the previous output frame image and to perform the temporal correction process for the target pixel, while switching between the correction filter and the another correction filter, according to the target pixel.

8. An image processing method for receiving and processing an input frame image captured by an imaging device to generate an output frame image, comprising:
aligning a previous input frame image that was input before a current input frame image that is to be a processing target or a previous output frame image generated from the previous input frame image with the current input frame image;
calculating a weighted average using a correction filter, wherein the weighted average is calculated by directly weighting a pixel value of a target pixel of the current input frame image and directly weighting pixel values of corresponding pixels of at least one of a plurality of previous input frame images that are a predetermined number of frames before the current input frame image;
performing a temporal correction process to correct the target pixel by using the weighted average and a pixel value of a corresponding pixel of one previous output frame image generated from any one of the plurality of previous input frame images; and
generating a current output frame image from the current input frame image by use of the current input frame image corrected by the temporal correction process,
wherein in the correction filter, weights on the pixel values of the corresponding pixels of the previous input frame images are set to decrease toward older previous input frame images, and
wherein in the correction filter, the weights on the pixel values of the corresponding pixels of the plurality of previous input frame images of the predetermined number of frames are set so that a difference from the weight on the pixel value of the corresponding pixel of the previous input frame image that is one frame after a given previous input frame image or from the weight on the pixel value of the target pixel is smaller than a difference between the weight on the pixel value of the corresponding pixel of the previous input frame image that is the predetermined number of frames ahead of the current input frame image and the weight on the pixel value of the corresponding pixel of the previous input frame image that is one frame before the previous input frame image that is the predetermined number of frames ahead of the current input frame image.

9. A non-transitory computer-readable recording medium storing an image processing program for letting a computer perform an image processing method to receive and process an input frame image captured by an imaging device to generate an output frame image, the image processing program letting the computer perform the image processing method comprising:
recording, in a storage, a recorded frame image comprising one of:
a previous input frame image that was input before a current input frame image that is to be a processing target; or
a previous output frame image generated from the previous input frame image;

aligning the current input frame image with the recorded frame image;
calculating a weighted average using a correction filter, wherein the weighted average is calculated by directly weighting a pixel value of a target pixel of the current input frame image and directly we

Assignees

• Morpho Inc

Inventors

• Satoh Masaki
• HIRAI Shun

Classifications

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N7/0132 Primary

  the field or frame frequency of the incoming video signal being multiplied by a positive integer, e.g. for flicker reduction · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/20201

  Motion blur correction · CPC title