Noise reconstruction for image denoising

Patent metadata

Field	Value
Publication number	US-12340488-B2
Application number	US-202217831578-A
Country	US
Kind code	B2
Filing date	Jun 3, 2022
Priority date	Dec 4, 2019
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

The present disclosure provides example apparatuses, methods, and devices for denoising an image. One example apparatus performs operations including receiving an input image. A trained artificial intelligence model is implemented to form an estimate of a noise pattern in the input image and form an output image by subtracting the estimate of the noise pattern from the input image, where the model is configured to form the estimate of the noise pattern, and the estimate of the noise pattern is representative of a noise pattern that is characteristic to a specific image sensor type.

First claim

The invention claimed is:

1. A computer-implemented method for training a model to perform noise reduction on images, the method comprising:
receiving a plurality of training images;
receiving a plurality of noise signatures; and
for each of the plurality of training images:
(i) selecting one of the plurality of noise signatures and applying that noise signature to the respective training image to form a noisy input image;
(ii) forming a first noise estimate in the noisy input image by implementing a candidate version of the model on the noisy input image and forming an estimate of the respective training image by subtracting the first noise estimate from the noisy input image;
(iii) forming a second noise estimate by implementing the candidate version of the model on the respective training image and the selected noise signature; and
(iv) adapting the candidate version of the model in dependence on (a) a difference between the respective training image and the estimate of the respective training image and (b) a difference between the second noise estimate and the selected noise signature.

2. The computer-implemented method as claimed in claim 1 , wherein the forming step (ii) is performed in a first pathway and the forming step (iii) is performed in a second pathway.

3. The computer-implemented method as claimed in claim 2 , wherein each of the first and second pathways comprises an encoder-decoder network.

4. The computer-implemented method as claimed in claim 3 , wherein a plurality of weights of a plurality of decoders of the first and second pathways are shared.

5. The computer-implemented method as claimed in claim 2 , wherein the first pathway and the second pathway are each based on a fully convolutional network.

6. The computer-implemented method as claimed in claim 2 , wherein the second pathway implements an unsupervised learning method.

7. The computer-implemented method as claimed in claim 2 , wherein the first pathway comprises one or more skip connections.

8. The computer-implemented method as claimed in claim 1 , wherein each of the plurality of training images is a RAW image or an RGB image.

9. The computer-implemented method as claimed in claim 1 , wherein the model is a convolutional neural network.

10. A device for training a model to perform noise reduction on images, comprising:
one or more processors; and
a non-transitory computer readable medium storing a program to be executed by the one or more processors, wherein the program comprises instructions that cause the device to perform operations comprising:
receiving a plurality of training images;
receiving a plurality of noise signatures; and
for each of the plurality of training images:
(i) selecting one of the plurality of noise signatures and applying that noise signature to the respective training image to form a noisy input image;
(ii) forming a first noise estimate in the noisy input image by implementing a candidate version of the model on the noisy input image and forming an estimate of the respective training image by subtracting the first noise estimate from the noisy input image;
(iii) forming a second noise estimate by implementing the candidate version of the model on the respective training image and the selected noise signature; and
(iv) adapting the candidate version of the model in dependence on (a) a difference between the respective training image and the estimate of the respective training image and (b) a difference between the second noise estimate and the selected noise signature.

11. The device as claimed in claim 10 , wherein the forming step (ii) is performed in a first pathway and the forming step (iii) is performed in a second pathway.

12. The device as claimed in claim 11 , wherein each of the first and second pathways comprises an encoder-decoder network.

13. The device as claimed in claim 12 , wherein a plurality of weights of a plurality of decoders of the first and second pathways are shared.

14. The device as claimed in claim 11 , wherein the first pathway and the second pathway are each based on a fully convolutional network.

Assignees

• Huawei Tech Co Ltd

Inventors

• Marras Ioannis
• Alexiou Ioannis
• Slabaugh Gregory
• Zafeiriou Stefanos

Classifications

• G06T2207/20224

  Image subtraction · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title