Motion-compensated temporal filtering based on variable filter parameters

Patent metadata

Field	Value
Publication number	US-9036695-B2
Application number	US-93828510-A
Country	US
Kind code	B2
Filing date	Nov 2, 2010
Priority date	Nov 2, 2010
Publication date	May 19, 2015
Grant date	May 19, 2015

Abstract

Systems and devices for, and methods of, motion-compensated temporal filtering based on variable filter parameters. A method embodiment includes (a) determining, by a processor having memory, a pixel-related residue image based on a set of differences between a current pixel intensity of a current frame and a corresponding pixel intensity of a previous frame, wherein the corresponding pixel intensity is augmented by a motion-compensated vector of the previous frame; (b) determining an intensity weight based on the determined pixel-related residue image and a temporal filtering parameter; and (c) filtering the pixel intensity of the current frame based on the determined intensity weight and the motion compensated vector of the previous frame.

First claim

What is claimed is:

1. A method of motion-compensated temporal filtering comprising:
determining, by a processor having memory, a pixel-related residue image based on a set of differences between a current pixel intensity of a current frame and a corresponding pixel intensity of a previous frame, wherein the corresponding pixel intensity is augmented by a motion-compensated vector of the previous frame;
determining an intensity weight based on the determined pixel-related residue image and a filtering parameter, wherein the filtering parameter provides, via minimizing a mean-square difference error, a de-noising effect based on noise level and motion compensation error;
generating an optimal filtering parameter for motion-compensated temporal filtering by subsequently revising the filtering parameter based on a set of previously determined residue images via tracking a count of frames and apportioning according to a resulting square-root of each residue image of the set of previously determined residue images;
filtering the pixel intensity of the current frame based on the determined intensity weight, the pixel intensity of the previous frame, a standard deviation parameter, and the motion compensated vector of the previous frame; and
determining a filtered pixel intensity vector based on the filtered pixel intensity of the current frame, the generated optimal filtering parameter, and a spatial weighting distribution.

2. The method of claim 1 wherein the filtering parameter is a temporal filtering parameter based on offline training.

3. The method of claim 2 wherein the temporal filtering parameter is further based on a histogrammed set of square-rooted determined pixel-related residue images.

4. The method of claim 3 wherein the histogrammed set of square-rooted determined pixel-related residue images is based on a mode of the histogrammed set of square-rooted determined pixel-related residue images.

5. The method of claim 4 wherein the histogrammed set of square-rooted determined pixel-related residue images is further based on a portion of the histogrammed set of square-rooted determined pixel-related residue images.

6. The method of claim 1 wherein the filtering parameter is a spatial statistical representation of at least one of: intensity and the motion vector.

7. The method of claim 6 wherein the filtering parameter is based on a variance of image intensity within a region associated with the current pixel.

8. The method of claim 7 wherein the filtering parameter is based on a variance of at least one directional component of a motion vector within a region associated with the current pixel.

9. The method of claim 8 wherein the filtering parameter is based on a variance of the current pixel intensity of the current frame and a variance of the motion vector within a region associated with the current pixel.

10. A method of motion-compensated temporal filtering comprising:
determining, by a processor having memory, a pixel-related residue image based on a set of differences between a current pixel intensity of a current frame and a corresponding pixel intensity of a previous frame, wherein the corresponding pixel intensity is augmented by a motion-compensated vector of the previous frame;
determining a set of intensity weights based on the determined pixel-related residue image and a temporal filtering parameter, wherein the filtering parameter provides, via minimizing a mean-square difference error, a de-noising effect based on noise level and motion compensation error;
determining a set of spatial weights based on a set of neighboring pixels;
determining an optimal filtering parameter via attenuating the determined set of intensity weights based on the determined spatial weights and tracking a count of frames and apportioning according to a resulting square-root of each residue image of a set of previously determined residue images;
filtering the pixel intensity of the current frame based on the set of determined intensity weight, the pixel intensity of the previous frame, a standard deviation parameter, the set of determined spatial weight, and the motion compensated vector of the previous frame; and
determining a filtered pixel intensity vector based on the filtered pixel intensity of the current frame, the determined optimal filtering parameter, and a spatial weighting distribution.

11. The method of claim 10 wherein the set of spatial weights is further based on a spatial candidate set comprising a plurality of motion compensated vectors of the previous frame.

12. The method of claim 11 wherein the spatial candidate set is further based on a set of neighboring pixels wherein the weight is attenuated distal from the pixel around which the intensity vector is being determined.

13. A device comprising:
a processor, configured to:
determine a pixel-related residue image based on a set of differences between a current pixel intensity of a current frame and a corresponding pixel intensity of a previous frame, the corresponding pixel intensity augmented by a motion-compensated vector of the previous frame;
determine an intensity weight based on the determined pixel-related residue image and a filtering parameter, wherein the filtering parameter provides, via minimizing a mean-square difference error, a de-noising effect based on noise level and motion compensation error;
generate an optimal filtering parameter for motion-compensated temporal filtering by subsequently revising the filtering parameter based on a set of previously determined residue images via tracking a count of frames and apportioning according to a resulting square-root of each residue image of the set of previously determined residue images;
filter the pixel intensity of the current frame based on the determined intensity weight, the pixel intensity of the previous frame, a standard deviation parameter, and the motion compensated vector of the previous frame; and
determine a filtered pixel intensity vector based on the filtered pixel intensity of the current frame, the generated optimal filtering parameter, and a spatial weighting distribution.

14. The device of claim 13 wherein the processor is further configured to determine a set of spatial weights based on a set of neighboring pixels, and is further configured to filter the pixel intensity of the current frame based on the determined intensity weight, the set of determined spatial weights, and the motion compensated vector of the previous frame.

15. The device of claim 13 wherein the filtering parameter is a temporal filtering parameter and the processor is further configured to determine the temporal filtering parameter based on the determined set of residue images.

16. The device of claim 15 wherein the temporal filtering parameter is further based on a histogrammed set of square-rooted determined pixel-related residue images.

17. The device of claim 16 wherein the histogrammed set of square-rooted determined pixel-related residue images is based on a mode of the histogrammed set of square-rooted determined pixel-related residue images.

18. The device of claim 17 wherein the histogrammed set of square-rooted determined pixel-related residue images is further based on a portion of the histogrammed set of square-rooted determined pixel-related residue images.

19. The device of claim 13 wherein the processor is further configured to determine the filtering parameter based on a spatial statistical representation of at least one of: intensity and the motion vector.

20. T

Assignees

• Yang Junlan
• Su Yeping
• Segall Christopher A
• Sharp Lab Of America Inc

Inventors

• Yang Junlan
• Su Yeping
• Segall Christopher A

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• G06T5/002 Primary

  Physics · mapped topic

• H04N19/139

  Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability · CPC title

• H04N19/615 Primary

  using motion compensated temporal filtering [MCTF] · CPC title