Low complexity adaptive filtering for mobile captures

Patent metadata

Field	Value
Publication number	US-9870598-B2
Application number	US-201313956735-A
Country	US
Kind code	B2
Filing date	Aug 1, 2013
Priority date	Apr 26, 2013
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

A method of noise filter parameter adaptation, the method comprising receiving a current video frame comprising a plurality of pixels. A table lookup is performed, using current statistical values associated with the current video frame. Noise filter parameters are adapted, based on current lighting conditions as determined from the performed table lookup. The current lighting conditions correspond to the current statistical values. The current video frame is noise filtered as defined by the adapted noise filter parameters.

First claim

What is claimed is:

1. A method of noise filtering, the method comprising:
receiving a video frame comprising video data for a plurality of pixels;
performing a table lookup in a table to determine a lighting condition for the video frame, wherein the table comprises a plurality of lighting conditions, wherein each of the plurality of lighting conditions in the table corresponds to a particular set of statistical values, wherein the table lookup is performed using statistical values generated from the video data in the video frame, and wherein the table lookup comprises using the statistical values generated from the video data to select, from the plurality of lighting conditions in the table, the lighting condition for the video frame;
adapting spatial noise filter parameters based on the lighting condition determined from the table lookup; and
performing spatial noise filtering of the video data as defined by the adapted spatial noise filter parameters to produce spatially filtered video data.

2. The method of claim 1 , further comprising:
adapting temporal noise filter parameters based on the lighting condition; and
performing temporal noise filtering of the spatially filtered video data as defined by the adapted temporal noise filter parameters.

3. The method of claim 2 further comprising:
adapting the temporal noise filtering parameters also based on a measure of a determined amount of motion in the spatially filtered video data.

4. The method of claim 3 further comprising combining an output of the spatial noise filtering and an output of the temporal noise filtering into a noise filtered output, wherein the output of the spatial noise filtering and the output of the temporal noise filtering are individually weighted according to the measure of the determined amount of motion and based on the lighting condition.

5. The method of claim 1 , wherein the lighting condition is selected from the group consisting of: indoor light, outdoor light, low lighting level, medium lighting level, and high lighting level.

6. The method of claim 1 , wherein the statistical values are received from an image signal processor.

7. The method of claim 1 , wherein the statistical values comprise at least two of the following:
lux values;
exposure values;
white balancing gains; and
correlated color temperature.

8. The method of claim 1 , wherein the current lighting condition is for an entire video frame.

9. A method of noise filtering, the method comprising:
performing a table lookup in a table to determine a lighting condition for a video frame, wherein the table comprises a plurality of lighting conditions, wherein each of the plurality of lighting conditions in the table corresponds to a particular set of statistical values, wherein the table lookup is performed using statistical values generated by an image signal processor from the video data in the video frame, and wherein the table lookup comprises using the statistical values associated with the video frame to select, from the plurality of lighting conditions in the table, the lighting condition for the video frame;
adapting noise filter parameters for each pixel of a first portion of pixels of the video frame based on the lighting condition and on a determined amount of motion in each pixel of the first portion of pixels;
borrowing adapted noise filter parameters for each pixel of a second portion of pixels of the video frame from a respective pixel of neighboring pixels that are part of the first portion of pixels of the video frame; and
noise filtering each pixel of the video frame as defined by each pixel's corresponding adapted noise filter parameters.

10. The method of claim 9 , wherein the pixels of the first portion of pixels and the second portion of pixels are arranged in a spatial pattern so that each pixel of the second portion of pixels is neighboring a pixel of the first portion of pixels.

11. The method of claim 9 , wherein the pixels of the first portion of pixels and the pixels of the second portion of pixels are arranged into a temporal pattern, wherein the temporal pattern changes with each successive video frame so that the first portion of pixels comprises different pixels for each video frame such that the noise filtering parameters for each pixel of the video frame will be adapted once every determined number of video frame cycles.

12. The method of claim 9 , wherein a ratio of the first portion of pixels to the second portion of pixels is dependent upon video content in the video frame and the lighting condition.

13. The method of claim 9 , wherein a ratio of the first portion of pixels to the second portion of pixels is selectable to bring one or more of power and computational budgets below a threshold.

14. The method of claim 9 , wherein the statistical values comprise at least two of the following:
lux values;
exposure values;
white balancing gains; and
correlated color temperature.

15. An apparatus comprising:
a video sensor configured to generate a video frame comprising video data for a plurality of pixels;
an image signal processor coupled to the video sensor and configured to generate statistical values based on the video frame; and
a memory coupled to the image signal processor that stores a plurality of lighting conditions, each of the lighting conditions related to corresponding statistical values, wherein statistical values associated with the video frame are used to select a lighting condition from the plurality of lighting conditions;
the apparatus configured to combine an output from a spatial noise filter and an output from a temporal noise filter for a particular pixel of the plurality of pixels, wherein the output from the spatial noise filter and the output from the temporal noise filter are individually weighted according to the particular pixel's level of motion and by the lighting condition;
wherein a spatial noise filter is configured by the apparatus to filter the video data as defined by spatial noise filter parameters that are selected based on the lighting condition.

16. The apparatus of claim 15 , wherein the temporal noise filter is configured by the apparatus to filter the video data as defined by temporal noise filter parameters that are determined based on the lighting condition.

17. The apparatus of claim 16 , further configured to classify each pixel of the plurality of pixels according to each pixel's level of motion, wherein the temporal noise filter parameters for the particular pixel of the plurality of pixels are adapted based on the particular pixel's level of motion.

18. The apparatus of claim 15 , wherein the lighting condition is selected from the group consisting of: indoor light, outdoor light, low lighting level, medium lighting level, and high lighting level.

19. The apparatus of claim 15 , wherein the statistical values comprise at least two of the following:
lux values;
exposure values;
white balancing gains; and
correlated color temperature.

Assignees

• Nvidia Corp

Inventors

• Allagadapa Varun Kumar
• Avadhanam Niranjan
• Kottana Thrinadh
• Gupta Shalini

Classifications

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title