Systems and methods for image denoising using deep convolutional networks

Patent metadata

Field	Value
Publication number	US-11508037-B2
Application number	US-202017010670-A
Country	US
Kind code	B2
Filing date	Sep 2, 2020
Priority date	Mar 10, 2020
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

A method for denoising an image includes: receiving, by a processing circuit of a user equipment, an input image; supplying, by the processing circuit, the input image to a trained convolutional neural network (CNN) including a multi-scale residual dense block (MRDB), the MRDB including: a residual dense block (RDB); and an atrous spatial pyramid pooling (ASPP) module; computing, by the processing circuit, an MRDB output feature map using the MRDB; and computing, by the processing circuit, an output image based on the MRDB output feature map, the output image being a denoised version of the input image.

First claim

What is claimed is:

1. A method for denoising an image comprises:
receiving, by a processing circuit of a user equipment, an input image;
supplying, by the processing circuit, the input image to a trained convolutional neural network (CNN) comprising a multi-scale residual dense block (MRDB), the MRDB comprising:
a residual dense block (RDB) comprising a plurality of convolutional modules; and
an atrous spatial pyramid pooling (ASPP) module;

computing, by the processing circuit, an MRDB output feature map based at least on:
an intermediate feature map computed by the plurality of convolutional modules of the RDB of the MRDB; and
an output of the ASPP module of the MRDB; and

computing, by the processing circuit, an output image based on the MRDB output feature map, the output image being a denoised version of the input image.

2. The method of claim 1 , further comprising supplying an input feature map to the MRDB,
wherein the input feature map is supplied to a cascade of the convolutional modules of the RDB to compute the intermediate feature map,
wherein the input feature map is supplied to the ASPP to compute a plurality of feature maps at different dilation rates,
wherein the plurality of feature maps at different dilation rates are concatenated by a concatenation layer,
wherein an output of the concatenation layer is concatenated with the intermediate feature map of the residual dense block to form an RDB output feature map, and
wherein the MRDB output feature map is computed based on the RDB output feature map.

3. The method of claim 2 , wherein the input feature map is supplied to an ASPP convolutional module, and
wherein the plurality of feature maps at different dilation rates are calculated based on an output of the ASPP convolutional module.

4. The method of claim 1 , wherein the trained CNN comprises a multi-scale residual dense network (MRDN) comprising one or more convolutional layers and a cascade of one or more MRDBs comprising the MRDB.

5. The method of claim 4 , wherein the input image is supplied to a first group of convolutional layers of the MRDN,
wherein an output of the first group of convolutional layers is supplied to the cascade of one or more MRDBs,
wherein a plurality of inputs to the one or more MRDBs are concatenated with the output of a last MRDB of the cascade of one or more MRDBs, compressed by a 1×1 convolutional layer, and supplied to a second group of convolutional layers to compute the MRDB output feature map,
wherein the MRDB feature map is added to an output of the second group of convolutional layers by an adder, and
wherein an output of the adder is supplied to a third group of convolutional layers to compute the output image.

6. The method of claim 1 , wherein the trained CNN comprises a first U-net with block connection (U-Net-B) network comprising an encoder and a decoder operating at a plurality of scales,
wherein a plurality of MRDBs comprising the MRDB connect the encoder and the decoder at the plurality of scales.

7. The method of claim 6 , wherein the trained CNN further comprises:
a second U-Net-B cascaded with the first U-Net-B to form a cascaded U-net with block connection (MCU-Net),
a first adder configured to add the input image to the output of the first U-Net-B, wherein the output of the first adder is connected to an input of the second U-Net-B; and
a second adder configured to add the output of the first adder to the output of the second U-Net-B, wherein the second adder is configured to compute the output of the CNN.

8. The method of claim 1 , wherein the trained CNN comprises a multi-scale residual dense network (MRDN) comprising one or more convolutional layers and a cascade of one or more MRDBs comprising the MRDB,
wherein the trained CNN further comprises a cascaded U-net with block connection (MCU-Net) comprising a first U-net with block connection (U-Net-B) network and a second U-Net-B,
wherein the MRDN and the MCU-Net are ensembled and configured to compute a first denoised image and a second denoised image, and
wherein the output image is a combination of the first denoised image and the second denoised image.

9. The method of claim 1 , wherein the user equipment further comprises a camera system integrated with the user equipment,
wherein the method further comprises controlling the camera system to capture the input image, and
wherein the input image is received by the processing circuit from the camera system.

10. A user equipment configured to denoise an image, the user equipment comprising:
a processing circuit; and
a memory storing instructions that, when executed by the processing circuit, cause the processing circuit to:
receive an input image;
supply the input image to a trained convolutional neural network (CNN) implemented by the processing circuit, the trained CNN comprising a multi-scale residual dense block (MRDB), the MRDB comprising:
a residual dense block (RDB) comprising a plurality of convolutional modules; and
an atrous spatial pyramid pooling (ASPP) module;

compute an MRDB output feature map based at least on:
an intermediate feature map computed by the plurality of convolutional modules of the RDB of the MRDB; and
an output of the ASPP module of the MRDB; and

compute an output image based on the MRDB output feature map, the output image being a denoised version of the input image.

11. The user equipment of claim 10 , wherein the memory further stores instructions that, when executed by the processing circuit, cause the processing circuit to supply an input feature map to the MRDB,
wherein the input feature map is supplied to a cascade of the convolutional modules of the RDB to compute the intermediate feature map,
wherein the input feature map is supplied to the ASPP to compute a plurality of feature maps at different dilation rates,
wherein the plurality of feature maps at different dilation rates are concatenated by a concatenation layer,
wherein an output of the concatenation layer is concatenated with the intermediate feature map of the residual dense block to form an RDB output feature map, and
wherein the MRDB output feature map is computed based on the RDB output feature map.

12. The user equipment of claim 11 , wherein the input feature map is supplied to an ASPP convolutional module, and
wherein the plurality of feature maps at different dilation rates are calculated based on an output of the ASPP convolutional module.

13. The user equipment of claim 10 , wherein the trained CNN comprises a multi-scale residual dense network (MRDN) comprising one or more convolutional layers and a cascade of one or more MRDBs comprising the MRDB.

14. The user equipment of claim 13 , wherein the input image is supplied to a first group of convolutional layers of the MRDN,
wherein an output of the first group of convolutional layers is supplied to the cascade of one or more MRDBs,
wherein a plurality of inputs to the one or more MRDBs are concatenated with the output of a last MRDB of the cascade of one or more MRDBs, compressed by a 1×1 convolutional layer, and supplied to a second group of convolutional layers to compute the MRDB output feature map,
wherein the MRDB feature map is added to an output of the second group of convolutional layers by an adder, and
wherein an output of the adder is supplied to a third group of convolutional layers to compute the output image.

15. The user equipment of claim 10 , wherein the trained CNN comprises a first U-net with block connection (U-Net-B) network comprising an encoder and a decoder operating at a plurality of scale

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yang Zengli
• Bao Long
• Wang Shuangquan
• Bai Dongwoon
• Lee Jungwon

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/048

  Activation functions · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06T5/30

  Erosion or dilatation, e.g. thinning · CPC title