Image restoration method and apparatus, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-2024378702-A1
Application number	US-202218691024-A
Country	US
Kind code	A1
Filing date	May 26, 2022
Priority date	Jan 4, 2022
Publication date	Nov 14, 2024
Grant date	—

Abstract

An image restoration method includes: acquiring an original low-light image to be restored; acquiring a pre-trained image processing model; and inputting the original low-light image into the image processing model, so that an optical feature extraction network in the image processing model extracts an illumination feature from the original low-light image, and an image feature extraction network extracts a target image feature from the original low-light image, and generating a target bright image based on the illumination feature and the target image feature. The optical feature extraction network and the image feature extraction network are respectively used to process an image feature of an original low-light image, so as to obtain an illumination feature and a target image feature, and the illumination feature is then fused with the target image feature for image restoration to obtain a bright image.

First claim

1 . An image restoration method, comprising:
acquiring an original low-light image to be restored; acquiring a pre-trained image processing model, wherein the image processing model comprises an optical feature extraction network and an image feature extraction network; and inputting the original low-light image into the image processing model, so that the optical feature extraction network in the image processing model extracts an illumination feature from the original low-light image, and the image feature extraction network extracts a target image feature from the original low-light image, and generating a target bright image based on the illumination feature and the target image feature.

2 . The method according to claim 1 , wherein acquiring a pre-trained image processing model comprises:
acquiring a training sample set, wherein the training sample set comprises a plurality of low-light sample images and a bright sample image corresponding to the low-light sample images; inputting the low-light sample image into an initial image processing model, so that an optical feature extraction network and an image feature extraction network in the initial image processing model respectively extract image features of the low-light sample image, and generating a bright image based on the image features; calculating a loss function value between the bright image and a bright sample image corresponding to the bright image; and when the loss function value is less than a preset threshold value, determining the initial image processing model as the image processing model.

3 . The method according to claim 2 , further comprising:
when the loss function value is greater than or equal to the preset threshold value, updating model parameters in the initial image processing model to obtain an updated initial image processing model; and training the updated initial image processing model using the low-light sample image in the training sample set until the loss function value between the bright image output by the updated initial image processing model and the bright sample image is less than the preset threshold value.

4 . The method according to claim 2 , wherein the initial image processing model comprises: the optical feature extraction network and the image feature extraction network, wherein the optical feature extraction network comprises a convolution parameter determined based on an illumination code matrix and a convolution layer, and the image feature extraction network comprises a plurality of fully connected layers.

5 . The method according to claim 4 , wherein before inputting the low-light sample image to an initial image processing model, the method further comprises:
acquiring a plurality of real bright images with different exposure degrees, and cropping the real bright images to obtain a plurality of image blocks; performing image encoding based on the image blocks to obtain code information, and obtaining the illumination code matrix according to the code information; and determining the convolution parameters of the optical feature extraction network in a feature fusion process based on the illumination code matrix, and determining channel coefficients of the image feature extraction network in the feature fusion process.

6 . The method according to claim 5 , wherein inputting the low-light sample image into an initial image processing model, so that the optical feature extraction network and the image feature extraction network in the initial image processing model extract image features of the low-light sample image, and generating a bright image based on the image features comprises:
inputting the low-light sample image to the initial image processing model, so that the initial image processing model extracts the image features of the low-light sample image, generating first image features according to the image features and the convolution parameters based on the optical feature extraction network, and generating second image features according to the image features and the channel coefficients based on the image feature extraction network, and fusing the first image features and the second image features to generate the bright image.

7 . (canceled)

8 . A non-transitory computer-readable storage medium, storing computer-readable instructions, wherein the computer-readable instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:
acquiring an original low-light image to be restored; acquiring a pre-trained image processing model, wherein the image processing model comprises an optical feature extraction network and an image feature extraction network; and inputting the original low-light image into the image processing model, so that the optical feature extraction network in the image processing model extracts an illumination feature from the original low-light image, and the image feature extraction network extracts a target image feature from the original low-light image, and generating a target bright image based on the illumination feature and the target image feature.

9 . An electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other via the communication bus; wherein:
the memory is used for storing computer-readable instructions; and the processor is used for executing operations comprising: acquiring an original low-light image to be restored; acquiring a pre-trained image processing model, wherein the image processing model comprises an optical feature extraction network and an image feature extraction network; and inputting the original low-light image into the image processing model, so that the optical feature extraction network in the image processing model extracts an illumination feature from the original low-light image, and the image feature extraction network extracts a target image feature from the original low-light image, and generating a target bright image based on the illumination feature and the target image feature.

10 . The method according to claim 1 , wherein inputting the original low-light image into the image processing model, so that the optical feature extraction network in the image processing model extracts an illumination feature from the original low-light image, and the image feature extraction network extracts a target image feature from the original low-light image, and generating a target bright image based on the illumination feature and the target image feature comprises:
inputting the original low-light image into the image processing model, so that the image processing model extracts image features from the original low-light image, generating the illumination feature according to the image features and an illumination code matrix based on the optical feature extraction network in the image processing model, generating the target image feature according to the image features and channel coefficients based on the image feature extraction network, and fusing the illumination feature and the target image feature to generate the target bright image.

11 . The method according to claim 1 , wherein acquiring an original low-light image to be restored comprises:
acquiring a low-light image from an image processing request sent by a client; and determining the low-light image as the original low-light image to be restored when a resolution of the low-light image is less than a preset resolution.

12 . The method according to claim 1 , wherein the optical feature extraction network comprises a co

Assignees

• Suzhou Metabrain Intelligent Technology Co Ltd

Inventors

• Zhang Yingjie
• Shi Hongzhi
• Zhao Yaqian

Classifications

• G06T2207/10024

  Color image · CPC title

• G06T5/60 Primary

  using machine learning, e.g. neural networks · CPC title

• G06T5/73

  Deblurring; Sharpening · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T2207/20081

  Training; Learning · CPC title