Deep unfolding algorithm for efficient image denoising under varying noise conditions

Patent metadata

Field	Value
Publication number	US-10043243-B2
Application number	US-201615176216-A
Country	US
Kind code	B2
Filing date	Jun 8, 2016
Priority date	Jan 22, 2016
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

A computer-implemented method for denoising image data includes a computer system receiving an input image comprising noisy image data and denoising the input image using a deep multi-scale network comprising a plurality of multi-scale networks sequentially connected. Each respective multi-scale network performs a denoising process which includes dividing the input image into a plurality of image patches and denoising those image patches over multiple levels of decomposition using a threshold-based denoising process. The threshold-based denoising process denoises each respective image patch using a threshold which is scaled according to an estimation of noise present in the respective image patch. The noising process further comprises the assembly of a denoised image by averaging over the image patches.

First claim

We claim:

1. A computer-implemented method for denoising image data, the method comprising:
receiving, a computer system, an input image comprising noisy image data;
denoising, by the computer system, the input image using a deep multi-scale network comprising a plurality of multi-scale networks sequentially connected, wherein each respective multi-scale network performs a denoising process comprising:
dividing the input image into a plurality of image patches,
denoising the plurality of image patches over multiple levels of decomposition using a threshold-based denoising process, wherein the threshold-based denoising process denoises each respective image patch using a threshold which is scaled according to an estimation of noise present in the respective image patch;
assembling a denoised image by averaging over the plurality of image patches.

2. The method of claim 1 , wherein the estimation of noise present in the respective image patch is based on a standard deviation of noise present in the respective image patch.

3. The method of claim 1 , wherein the threshold-based denoising process applied to each respective image patch comprises:
generating a plurality of non-zero coefficients providing a sparse representation of the respective image patch according to a predetermined dictionary;
applying the threshold to the plurality of non-zero coefficients to yield a plurality of thresholded coefficients; and
determining an inverse transform of the plurality of thresholded coefficients to yield reconstructed image data representative of the respective image patch.

4. The method of claim 3 , wherein the threshold is a garrote thresholding function parameterized by the noise level in the patch.

5. The method of claim 3 , wherein the threshold used by the respective multi-scale network is further scaled based on one of:
an initial noise estimate corresponding to the respective image patch, or
residual noise present in the respective image patch following processing by an immediately preceding multi-scale network in the deep multi-scale network.

6. The method of claim 3 , wherein the predetermined dictionary is learned using a K-SVD process using a plurality of training images.

7. The method of claim 1 , further comprising:
individually training each of the plurality of multi-scale networks using a plurality of training images.

8. The method of claim 7 , wherein each respective multi-scale network is trained by minimizing mean squared error (MSE) of the plurality of training images when processed by the respective multi-scale network.

9. The method of claim 7 , wherein each respective multi-scale network is trained by maximizing structural similarity (SSIM) of the plurality of training images when processed by the respective multi-scale network.

10. The method of claim 7 , wherein each respective multi-scale network is trained using a subset of the plurality of training images which exhibit low peak signal-to-noise when reconstructed by the respective multi-scale network in comparison to other images in the plurality of training images.

11. A system for denoising image data, the system comprising:
a neural network configured to denoise an image patch, the neural network comprising:
a first convolutional layer configured to perform a decomposition operation on the image patch to yield a plurality of coefficients;
a plurality of neurons configured to perform non-linear thresholding of the plurality of coefficients, wherein the plurality of neurons utilize a threshold which is scaled according to an estimation of noise present in the image patch;
a second convolutional layer configured to perform a reconstruction operation of the image patch on the plurality of coefficients following the non-linear thresholding;

a decomposition component configured to recursively utilize the neural network to recursively denoise subsampled representations of a noisy image using the neural network; and
an assembly component configured to assemble output of the neural network and the decomposition component into a denoised image.

12. The system of claim 11 , further comprising:
a plurality of processors configured to parallelize at least one of the decomposition operation, the non-linear thresholding, or the reconstruction operation performed by the neural network.

13. The system of claim 11 , further comprising a training component which is configured to train the neural network by simultaneously adjusting weights in all convolutional layers to minimize a loss function between ground truth clean training examples and counterpart examples artificially corrupted with noise.

14. The system of claim 13 , wherein mean squared error (MSE) is used as the loss function.

15. The system of claim 13 , wherein structural dissimilarity is used as the loss function.

16. The system of claim 11 , wherein the threshold is a garrote thresholding function.

17. The system of claim 11 , the threshold is further scaled based on on residual noise present in the respective image patch following processing by an immediately preceding execution of the neural network by the decomposition component.

18. An article of manufacture for denoising image data, the article of manufacture comprising a non-transitory, tangible computer-readable medium holding computer-executable instructions for performing a process comprising:
denoising an input image comprising noisy image data using a deep multi-scale network comprising a plurality of multi-scale networks sequentially connected, wherein each respective multi-scale network performs a denoising process comprising:
dividing the input image into a plurality of image patches,
denoising the plurality of image patches over multiple levels of decomposition using a threshold-based denoising process, wherein the threshold-based denoising process denoises each respective image patch using a threshold which is scaled according to an estimation of noise present in the respective image patch;
assembling a denoised image by averaging over the plurality of image patches.

19. The article of manufacture of claim 18 , wherein the threshold-based denoising process applied to each respective image patch comprises:
generating a plurality of non-zero coefficients providing a sparse representation of the respective image patch according to a predetermined dictionary;
applying the threshold to the plurality of non-zero coefficients to yield a plurality of thresholded coefficients; and
determining an inverse transform of the plurality of thresholded coefficients to yield reconstructed image data representative of the respective image patch.

20. The article of manufacture of claim 18 , wherein the threshold used by the respective multi-scale network is further scaled based on one of:
an initial noise estimate corresponding to the respective image patch, or
residual noise present in the respective image patch following processing by an immediately preceding multi-scale network in the deep multi-scale network.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Matviychuk Yevgen
• Mailhe Boris
• Chen Xiao
• Wang Qiu
• Nadar Mariappan S

Classifications

• G06V10/772

  Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06F18/21342

  using statistical independence, i.e. minimising mutual information or maximising non-gaussianity · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title