System and method for designing efficient super resolution deep convolutional neural networks by cascade network training, cascade network trimming, and dilated convolutions

Patent metadata

Field	Value
Publication number	US-11354577-B2
Application number	US-201816138279-A
Country	US
Kind code	B2
Filing date	Sep 21, 2018
Priority date	Mar 15, 2017
Publication date	Jun 7, 2022
Grant date	Jun 7, 2022

Abstract

Apparatuses and methods of manufacturing same, systems, and methods are described. In one aspect, a method includes generating a convolutional neural network (CNN) by training a CNN having three or more convolutional layers, and performing cascade training on the trained CNN. The cascade training includes an iterative process of one or more stages, in which each stage includes inserting a residual block (ResBlock) including at least two additional convolutional layers and training the CNN with the inserted ResBlock.

First claim

What is claimed is:

1. A method, comprising:
generating a convolutional neural network (CNN), wherein generating the CNN comprises:
training a CNN having three or more convolutional layers; and
performing cascade training on the trained CNN,

wherein cascade training comprises an iterative process of a plurality of stages, in which each of the stages comprises:
inserting a residual block (ResBlock) including at least two additional convolutional layers; and
training the CNN with the inserted ResBlock.

2. The method of claim 1 , wherein the inserted ResBlock further includes a rectified linear unit layer between the at least two additional convolutional layers.

3. The method of claim 1 , wherein each of the stages further comprises replacing one of the convolutional layers with a depthwise separable convolutional layer.

4. The method of claim 3 , wherein each of the stages further comprises initializing the depthwise separable convolutional layer with random weights.

5. The method of claim 4 , wherein each of the stages further comprises training the CNN with the replaced depthwise separable convolutional layer.

6. The method of claim 1 , wherein a weight of the inserted ResBlock is randomly initialized.

7. The method of claim 1 , wherein the CNN is trained on multiple color channels.

8. The method of claim 1 , wherein the cascade training further comprises replacing the ResBlock with a depthwise separable residual block (DS-ResBlock).

9. The method of claim 1 , further comprising denoising an image with the generated CNN.

10. The method of claim 9 , wherein denoising the image comprises applying an edge-aware loss function to the image.

11. An apparatus, comprising:
one or more non-transitory computer-readable media; and
at least one processor which, when executing instructions stored on the one or more non-transitory computer-readable media, performs the steps of:
generating a convolutional neural network (CNN) by:
training a CNN having three or more convolutional layers; and
performing cascade training on the trained CNN,

wherein cascade training comprises an iterative process of a plurality of stages, in which each of the stages comprises:

inserting a residual block (ResBlock) including at least two additional convolutional layers; and
training the CNN with the inserted ResBlock.

12. The apparatus of claim 11 , wherein the inserted ResBlock further includes a rectified linear unit layer between the at least two additional convolutional layers.

13. The apparatus of claim 11 , wherein each of the stages further comprises replacing one of the convolutional layers with a depthwise separable convolutional layer.

14. The apparatus of claim 13 , wherein each of the stages further comprises initializing the depthwise separable convolutional layer with random weights.

15. The apparatus of claim 14 , wherein of the stages stage further comprises training the CNN with the replaced depthwise separable convolutional layer.

16. The apparatus of claim 11 , wherein a weight of the inserted ResBlock is randomly initialized.

17. The apparatus of claim 11 , wherein the CNN is trained on multiple color channels.

18. The apparatus of claim 11 , wherein the cascade training further comprises replacing the ResBlock with a depthwise separable residual block (DS-ResBlock).

19. The apparatus of claim 11 , wherein the at least one processor, when executing the instructions, denoises an image with the generated CNN.

20. The apparatus of claim 19 , wherein denoising the image comprises applying an edge-aware loss function to the image.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Ren Haoyu
• El-Khamy Mostafa
• Lee Jungwon

Classifications

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title