Content adaptive motion compensated temporal filtering for denoising of noisy video for efficient coding

Patent metadata

Field	Value
Publication number	US-10448014-B2
Application number	US-201715603147-A
Country	US
Kind code	B2
Filing date	May 23, 2017
Priority date	May 23, 2017
Publication date	Oct 15, 2019
Grant date	Oct 15, 2019

Abstract

Techniques related to improved video denoising using content adaptive motion compensated temporal filtering are discussed. Such techniques may include determining whether a block of a video frame is motion compensable and, when the block is motion compensable, generating a denoised block corresponding to the block using the block itself and averaged reference blocks from two or more motion compensation reference frames.

First claim

What is claimed is:

1. A computer implemented method for reducing noise in video comprising:
receiving a video frame having a target block and two or more second blocks that neighbor or overlap the target block;
performing motion estimation and compensation to determine, for the target block, a first motion compensated block from a first reference frame and a second motion compensated block from a second reference frame and to determine, for the target block, a plurality of reference blocks, each reference block comprising a block of the first or second reference frame that is overlapped when a motion vector of the two or more second blocks is used to translate the target block to the first or second reference frame;
when the target block is motion compensable, generating a denoised block corresponding to the target block based on the target block, the first motion compensated block, the second motion compensated blocks, and the reference blocks by:
when motion between the target block and the first and second motion compensated blocks and between the second blocks and the reference blocks is small motion, performing pixel wise median filtering based on the target block, a first average block of the first motion compensated block and reference blocks from the first reference frame, and a second average block of the second motion compensated block and reference blocks from the second reference frame to generate a median filtered block and determining a weighted average of the target block and the median filtered block to generate the denoised block; and
when motion between the target block and the first and second motion compensated blocks and between the second blocks and the reference blocks is not small motion, determining a weighted average of the target block, the first average block, and the second average block; and

outputting a denoised video frame comprising the denoised block.

2. The method of claim 1 , further comprising:
averaging magnitudes of motion vectors between the target block and the first motion compensated block, the target block and the second motion compensated block, and the second blocks and the reference blocks to determine an average motion vector size; and
comparing the average motion vector size to a threshold to determine whether the motion between the target block and the first and second motion compensated blocks and between the second blocks and the reference blocks is small motion or not small motion.

3. The method of claim 1 , wherein performing motion estimation and compensation comprises:
performing overlapping motion estimation based on the video frame and the first reference video frame to determine a first motion vector between the target block and the first motion compensated block and a second motion vector between one of the second blocks and a motion estimation block of the first reference frame; and
selecting a portion of the motion estimation block overlapping a translation of the target block to the first reference frame as one of the reference blocks.

4. The method of claim 1 , wherein performing motion estimation and compensation comprises:
performing non-overlapping motion estimation based on the video frame and the first reference video frame to determine a first motion vector between the target block and the first motion compensated block and a second motion vector between one of the second blocks and a motion estimation block of the first reference frame; and
extending the motion estimation block in the first reference frame to overlap a translation of the target block to the first reference frame and selecting the extended portion of the first reference frame as one of the reference blocks.

5. The method of claim 1 , further comprising:
deblock filtering the denoised frame to generate a denoised and deblocked frame and at least one of presenting the denoised and deblocked frame to a user or encoding the denoised and deblocked frame to generate a bitstream.

6. The method of claim 1 , wherein the video frame and the reference frame are top or bottom fields of an interlaced video frame.

7. The method of claim 1 , further comprising:
determining whether the target block is motion compensable based at least on motion between the target block and the first and second motion compensated blocks and comparisons of the target block to the first and second motion compensated blocks; and
when the target block is not motion compensable, generating the denoised block corresponding to the target block based on a spatial filtering of the target block.

8. A system for reducing noise in video comprising:
a memory configured to store a video frame having a target block and two or more second blocks that neighbor or overlap the target block; and
a processor coupled to the memory, the processor to:
perform motion estimation and compensation to determine, for the target block, a first motion compensated block from a first reference frame and a second motion compensated block from a second reference frame and to determine, for the target block, a plurality of reference blocks, each reference block comprising a block of the first or second reference frame that is overlapped when a motion vector of the two or more second blocks is used to translate the target block to the first or second reference frame,
when the target block is motion compensable, generate a denoised block corresponding to the target block based on the target block, the first motion compensated block, the second motion compensated blocks, and the reference blocks by being configured to:
when motion between the target block and the first and second motion compensated blocks and between the second blocks and the reference blocks is small motion, perform pixel wise median filtering based on the target block, a first average block of the first motion compensated block and reference blocks from the first reference frame, and a second average block of the second motion compensated block and reference blocks from the second reference frame to generate a median filtered block and determine a weighted average of the target block and the median filtered block to generate the denoised block; and
when motion between the target block and the first and second motion compensated blocks and between the second blocks and the reference blocks is not small motion, determine a weighted average of the target block, the first average block, and the second average block; and

output a denoised video frame comprising the denoised block.

9. The system of claim 8 , the processor further to:
average magnitudes of motion vectors between the target block and the first motion compensated block, the target block and the second motion compensated block, and the second blocks and the reference blocks to determine an average motion vector size; and
compare the average motion vector size to a threshold to determine whether the motion between the target block and the first and second motion compensated blocks and between the second blocks and the reference blocks is small motion or not small motion.

10. The system of claim 8 , the processor further to:
determine whether the target block is motion compensable based at least on motion between the target block and the first and second motion compensated blocks and comparisons of the target block to the first and second motion compensated blocks; and
when the target block is not motion compensable, generate the denoised block corresponding to the target block based on a spatial filtering of the target block.

11. At least one non-transitory machine readable medium comprising a plurality of instructions that, in response to being executed on a device, cause the de

Assignees

• Intel Corp

Inventors

• Possos Sebastian
• Puri Atul

Classifications

• H04N19/137

  Motion inside a coding unit, e.g. average field, frame or block difference · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/117 Primary

  Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) · CPC title

• H04N19/583

  Motion compensation with overlapping blocks · CPC title

• H04N19/86

  involving reduction of coding artifacts, e.g. of blockiness · CPC title