Low complexity adaptive temporal filtering for mobile captures

Patent metadata

Field	Value
Publication number	US-2016284060-A1
Application number	US-201313956735-A
Country	US
Kind code	A1
Filing date	Aug 1, 2013
Priority date	Apr 26, 2013
Publication date	Sep 29, 2016
Grant date	—

Abstract

A method of noise filter parameter adaptation, the method comprising receiving a current video frame comprising a plurality of pixels. A table lookup is performed, using current statistical values associated with the current video frame. Noise filter parameters are adapted, based on current lighting conditions as determined from the performed table lookup. The current lighting conditions correspond to the current statistical values. The current video frame is noise filtered as defined by the adapted noise filter parameters.

First claim

What is claimed is:

1 . A method of noise filter parameter adaptation, the method comprising:
receiving a current video frame comprising a plurality of pixels; performing a table lookup using current statistical values associated with the current video frame; adapting noise filter parameters based on current lighting conditions as determined from the performed table lookup, wherein the current lighting conditions correspond to the current statistical values; and noise filtering the plurality of pixels as defined by the adapted noise filter parameters.

2 . The method of claim 1 , wherein the noise filtering comprises:
spatial noise filtering the plurality of pixels as defined by the adapted noise filter parameters; and temporal noise filtering the plurality of pixels as defined by the adapted noise filter parameters, and wherein the adapted noise filter parameters comprise adapted noise filter parameters for each pixel of the plurality of pixels.

3 . The method of claim 1 further comprising:
adapting the noise filter parameters for a particular pixel of the plurality of pixels based on a determined amount of motion in the particular pixel.

4 . The method of claim 3 further comprising combining a spatial noise filtering output and a temporal noise filtering output into a noise filtered output for a particular pixel, and wherein an amount of the temporal noise filtering to be included in the noise filtered output is weighted as controlled by the determined amount of motion in the particular pixel and by the current lighting conditions.

5 . The method of claim 1 , wherein the table comprises a plurality of characterized lighting conditions, wherein each of the plurality of characterized lighting conditions in the table corresponds to a particular set of statistical values.

6 . The method of claim 1 , wherein the current lighting conditions comprise:
lighting sources comprising indoor light and outdoor light; and lighting levels comprising low, medium, and high lighting levels.

7 . The method of claim 1 , wherein the statistical values are received from an image signal processor.

8 . The method of claim 1 , wherein the statistical values comprise at least two or more of the following:
lux; exposure values; white balancing gains; and correlated color temperature.

9 . The method of claim 1 , wherein the current lighting conditions are for an entire current video frame.

10 . A method of noise filter parameter adaptation, the method comprising:
adapting noise filter parameters for each pixel of a first portion of pixels of a current video frame based on current lighting conditions and on a determined amount of motion in each pixel of the first portion of pixels; borrowing adapted noise filter parameters for each pixel of a second portion of pixels of the current video frame from one of neighboring pixels that are part of the first portion of pixels, and co-located pixels from a previous frame; and noise filtering each pixel of the current video frame as defined by their corresponding adapted noise filter parameters.

11 . The method of claim 10 , wherein the pixels of the first portion of pixels and the second portion of pixels are arranged in a spatial pattern so that each pixel of the second portion of pixels is neighboring a pixel of the first portion of pixels.

12 . The method of claim 10 , wherein the pixels of the first portion of pixels and the pixels of the second portion of pixels are arranged into a temporal pattern, wherein the temporal pattern changes with each successive video frame so that the first portion of pixels comprises different pixels each video frame such that the noise filtering parameters for each pixel of the video frame will be adapted once every determined number of video frame cycles.

13 . The method of claim 10 , wherein a ratio of the first portion of pixels to the second portion of pixels is dependent upon current video content and current lighting conditions.

14 . The method of claim 10 , wherein a ratio of the first portion of pixels to the second portion of pixels is selectable to bring one or more of power and computational budgets below a threshold.

15 . An apparatus for adapting noise filter parameters, the apparatus comprising:
a video sensor operable to generate a video frame comprising a plurality of pixels; an image signal processor operable to generate statistical values; a memory comprising a plurality of lighting conditions, each related to corresponding statistical values, wherein current statistical values associated with a current video frame are used to select a current lighting condition from the plurality of lighting conditions; and a noise filter operable to filter noise from the plurality of pixels as defined by adapted noise filter parameters, wherein the noise filter parameters are adapted based on the current lighting conditions.

16 . The apparatus of claim 15 , wherein the noise filter comprises:
a spatial filter operable to noise filter the plurality of pixels as defined by the adapted noise filter parameters; and a temporal filter operable to noise filter the plurality of pixels as defined by the adapted noise filter parameters, and wherein the adapted noise filter parameters comprise adapted noise filter parameters for each pixel of the plurality of pixels.

17 . The apparatus of claim 15 further comprising:
a motion classifier operable to classify motion in each pixel of the plurality of pixels, wherein noise filter parameters of the temporal filter for a particular pixel of the plurality of pixels are adapted based on an amount of motion in the particular pixel, as determined by the motion classifier.

18 . The apparatus of claim 17 further comprising:
an output combiner operable to combine an output from the spatial noise filter and an output from the temporal noise filter for a particular pixel, wherein an amount of the temporal noise filtering to be included in the noise filtered output is weighted by the output combiner as controlled by the amount of motion in the particular pixel and by the current lighting conditions.

19 . The apparatus of claim 15 , wherein the current lighting conditions comprise:
lighting sources comprising indoor light and outdoor light; and lighting levels comprising low, medium, and high lighting levels.

20 . The apparatus of claim, wherein the statistical values comprise at least two or more of the following:
lux; exposure values; white balancing gains; and correlated color temperature.

Assignees

• Nvidia Corp

Inventors

• Allagadapa Varun Kumar
• Avadhanam Niranjan
• Kottana Thrinadh
• Gupta Shalini

Classifications

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06T7/2033

  Physics · mapped topic

• G06T2207/20024

  Filtering details · CPC title

• G06K9/00335

  Physics · mapped topic