Image super-resolution method, image super-resolution device, and computer readable storage medium

Patent metadata

Field	Value
Publication number	US-10991076-B2
Application number	US-201916545967-A
Country	US
Kind code	B2
Filing date	Aug 20, 2019
Priority date	Feb 11, 2018
Publication date	Apr 27, 2021
Grant date	Apr 27, 2021

Abstract

Disclosed is an image super-resolution method, which includes: acquiring and amplifying an image to be processed, and extracting a scaling feature from the amplified image, to obtain a first image to be processed; sending the first image to be processed to a residual network, for the residual network outputting a corrected second image to be processed; and restoring the second image to be processed to generate a restored image, and outputting the restored image. The present disclosure further provides an image super-resolution device and a computer readable storage medium.

First claim

We claim:

1. An image super-resolution method, wherein the method comprises:
acquiring and amplifying an image to be processed, and extracting a scaling feature from the amplified image, to obtain a first image to be processed;
sending the first image to be processed to a residual network, for the residual network outputting a corrected second image to be processed; and
restoring the second image to be processed to generate a restored image, and outputting the restored image, wherein
the operation of acquiring and amplifying an image to be processed, and extracting a scaling feature from the amplified image, to obtain a first image to be processed, comprises:
acquiring a low-resolution image to be processed, and pre-processing the image to be processed in a pre-processing convolution layer; and
sending the pre-processed image to be processed to a scale amplification module, amplifying the image to be processed based on a preset amplification scale, and extracting the scaling feature from the amplified image, to obtain the first image to be processed, and
the preset amplification scale is defined as two times, three times, or four times.

2. The method according to claim 1 , wherein the operation of sending the first image to be processed to a residual network, for the residual network outputting a corrected second image to be processed, comprises:
sending the first image to be processed to the residual network, processing the first image to be processed by a plurality of bottleneck residual units in the residual network, to generate the corrected second image to be processed; and
sending the second image to be processed to a scale restoring module.

3. The method according to claim 2 , wherein the residual network comprises the plurality of bottleneck residual units and a convolution layer, and each bottleneck residual unit is connected with a weight normalization module.

4. The method according to claim 3 , wherein the bottleneck residual unit comprises three convolution layers, and an activation function layer is defined between each two adjacent convolution layers, and the activation function is a Parametric Rectified Linear Unit function.

5. The method according to claim 4 , wherein the activation function comprises a variable, and the value of the variable is obtained through learning from an upper network layer.

6. The method according to claim 1 , wherein the operation of restoring the second image to be processed to generate a restored image, and outputting the restored image, comprises:
on condition that a scale restoring module receives the second image to be processed, reducing the scale of the second image to be processed based on a scale in a scale amplification module, to generate the restored image; and
outputting the restored image.

7. The method according to claim 1 , wherein the operation of sending the first image to be processed to a residual network, for the residual network outputting a corrected second image to be processed, comprises:
sending the first image to be processed to the residual network, processing the first image to be processed by a plurality of bottleneck residual units in the residual network, to generate the corrected second image to be processed; and
sending the second image to be processed to a scale restoring module.

8. An image super-resolution device, wherein the device comprises: a memory, a processor, and an image super-resolution program stored on the memory and executable on the processor, the program, when executed by the processor, implements the following operations:
acquiring and amplifying an image to be processed, and extracting a scaling feature from the amplified image, to obtain a first image to be processed;
sending the first image to be processed to a residual network, for the residual network outputting a corrected second image to be processed; and
restoring the second image to be processed to generate a restored image, and outputting the restored image, wherein
the operation of acquiring and amplifying an image to be processed, and extracting a scaling feature from the amplified image, to obtain a first image to be processed, comprises:
acquiring a low-resolution image to be processed, and pre-processing the image to be processed in a pre-processing convolution layer; and
sending the pre-processed image to be processed to a scale amplification module, amplifying the image to be processed based on a preset amplification scale, and extracting the scaling feature from the amplified image, to obtain the first image to be processed, and
the preset amplification scale is defined as two times, three times, or four times.

9. The device according to claim 8 , wherein the program, when executed by the processor, implements the following operations:
sending the first image to be processed to the residual network, processing the first image to be processed by a plurality of bottleneck residual units in the residual network, to generate the corrected second image to be processed; and
sending the second image to be processed to a scale restoring module.

10. A non-transitory computer readable storage medium, wherein an image super-resolution program is stored on the non-transitory computer readable storage medium, the program, when executed by a processor, implements the following operations:
acquiring and amplifying an image to be processed, and extracting a scaling feature from the amplified image, to obtain a first image to be processed;
sending the first image to be processed to a residual network, for the residual network outputting a corrected second image to be processed; and
restoring the second image to be processed to generate a restored image, and outputting the restored image, wherein
the operation of acquiring and amplifying an image to be processed, and extracting a scaling feature from the amplified image, to obtain a first image to be processed, comprises:
acquiring a low-resolution image to be processed, and pre-processing the image to be processed in a pre-processing convolution layer; and
sending the pre-processed image to be processed to a scale amplification module, amplifying the image to be processed based on a preset amplification scale, and extracting the scaling feature from the amplified image, to obtain the first image to be processed, and
the preset amplification scale is defined as two times, three times, or four times.

11. The non-transitory computer readable storage medium according to claim 10 , wherein the program, when executed by the processor, implements the following operations:
sending the first image to be processed to the residual network, processing the first image to be processed by a plurality of bottleneck residual units in the residual network, to generate the corrected second image to be processed; and
sending the second image to be processed to a scale restoring module.

Assignees

• Shenzhen Skyworth Rgb Electronics Co Ltd

Inventors

• Huang Zhe
• Xiao Zhilin

Classifications

• G06T3/4076 Primary

  using the original low-resolution images to iteratively correct the high-resolution images · CPC title

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T3/4046

  using neural networks · CPC title

• G06T2207/20081

  Training; Learning · CPC title