Structure aware image denoising and noise variance estimation

Patent metadata

Field	Value
Publication number	US-9852353-B2
Application number	US-201414539767-A
Country	US
Kind code	B2
Filing date	Nov 12, 2014
Priority date	Nov 12, 2014
Publication date	Dec 26, 2017
Grant date	Dec 26, 2017

Abstract

Structure aware image denoising and noise variance estimation techniques are described. In one or more implementations, structure-aware denoising is described which may take into account a structure of patches as part of the denoising operations. This may be used to select one or more reference patches for a pixel based on a structure of the patch, may be used to compute weights for patches that are to be used to denoised a pixel based on similarity of the patches, and so on. Additionally, implementations are described to estimate noise variance in an image using a map of patches of an image to identify regions having pixels having a variance that is below a threshold. The patches from the one or more regions may then be used to estimate noise variance for the image.

First claim

What is claimed is:

1. A method comprising:
denoising, by at least one computing device, an image, the denoising including for a pixel in the image:
forming a plurality of potential reference patches from the image in which each potential reference patch of the plurality of potential reference patches includes the pixel;
selecting a reference patch from the plurality of potential reference patches, the selecting based at least in part on uniformity within each potential reference patch of the plurality of potential reference patches, the selecting comprising:
determining entropy of the plurality of potential reference patches; and
selecting the reference patch such that the reference patch has a lower entropy than others of the plurality of potential reference patches that include the pixel and that are not selected as the reference patch; and
computing a display value for the pixel in the image through comparison of the reference patch with a plurality of query patches; and

outputting, by the at least one computing device, a denoised image having the display value for the pixel.

2. A method as described in claim 1 , wherein the display value for the pixel specifies intensity or color of the pixel.

3. A method as described in claim 1 , wherein the selecting is performed based at least in part on calculating a distance between the potential reference patches indicating similarity of the potential reference patches, one to another, and determining that the reference patch has a lowest distance to a nearby other potential reference patch.

4. A method as described in claim 1 , wherein the selecting is performed based at least in part on a local structure prediction or gradient.

5. A method as described in claim 1 , wherein the denoising is performed as part of a non-local means (NLM) algorithm.

6. A method as described in claim 1 , wherein the display value for the pixel is further computed through comparison of at least one other reference patch with the plurality of potential reference patches, the at least one other reference patch selected from the plurality of potential reference patches.

7. A non-transitory system comprising:
at least one processor; and
at least one module implemented at least partially in hardware that, responsive to execution by the at least one processor, causes the at least one processor to perform structurally-aware denoising on an image to form a denoised image by:
forming a plurality of potential reference patches from the image in which each potential reference patch of the plurality of potential reference patches includes a pixel;
selecting a reference patch from the plurality of potential reference patches, the selecting based at least in part on the reference patch having a least amount of structure within the plurality of potential reference patches;
computing a display value for the pixel to be included in the denoised image using a weighted comparison of the reference patch with a plurality of query patches; and
outputting the denoised image having the computed display value for the pixel in the image.

8. A system as described in claim 7 , wherein the selecting is performed based at least in part on determining that the reference patch has a lowest distance to a nearby potential reference patch of the plurality of potential reference patches, the distance indicating similarity of the potential reference patches, one to another.

9. A system as described in claim 7 , wherein the selecting is performed based at least in part on a local structure prediction.

10. A system as described in claim 7 , wherein the selecting is performed based at least in part on gradient.

11. A system as described in claim 7 , wherein the selecting is performed based at least in part on determining that the reference patch has a lowest entropy of the plurality of potential reference patches.

12. A method of estimating noise variance in an image by at least one computing device for use in denoising the image, the method comprising:
generating, by the at least one computing device, a denoised image by applying a patch-based denoising algorithm to the image;
forming a map, by the at least one computing device, as having plurality of patches taken from the image, each of the plurality of patches including multiple pixels;
identifying one or more regions of the image by the at least one computing device, each of the one or more regions comprising two or more patches of the plurality of patches in the map, the one or more regions identified based on the multiple pixels included in each of the plurality of patches having a variance, one to another, that is below a threshold, the variance indicative of a uniformity of the multiple pixels included in each of the plurality of patches;
estimating the noise variance by the at least one computing device for the image, the estimated noise variance based on the two or more patches identified in the one or more regions in the image;
using the estimated noise variance to calculate a value for at least one pixel in the image; and
outputting the image with the calculated value for the at least one pixel.

13. A method as described in claim 12 , wherein the identifying includes measuring a patch uniformity score for each of the plurality of patches and generating a uniformity score map that is usable to identify the one or more regions.

14. A method as described in claim 13 , wherein the measuring of the patch uniformity score is computed based on pixel sample variance.

15. A method as described in claim 13 , wherein the measuring of the patch uniformity score is computed using a linearly combined function of the variance and a content metric.

16. A method as described in claim 13 , wherein the identifying includes discarding at least one region that has a size that is below a threshold.

17. A method as described in claim 12 , wherein each of the one or more regions includes a continuous collection of respective patches.

18. A method as described in claim 12 , wherein the estimating is computed as a median among each squared sum of patches within the one or more regions.

19. A method as described in claim 12 , wherein the variance is further indicative of the two or more patches in the one or more regions being relatively uniform and structure-less.

20. A method as described in claim 13 , further comprising comparing each of the uniformity scores to a fixed percentage parameter to select a set of the highest uniformity scored patches for the estimating the noise variance.

Assignees

• Adobe Systems Inc

Inventors

• Lin Zhe
• Kim Byungmoon
• Gao Yuan

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06K2009/4666

  Physics · mapped topic

• G06K9/46

  Physics · mapped topic

• G06K9/52 Primary

  Physics · mapped topic

• G06K9/6215

  Physics · mapped topic