Method and system for reconstructing super-resolution image

Patent metadata

Field	Value
Publication number	US-10181092-B2
Application number	US-201715481430-A
Country	US
Kind code	B2
Filing date	Apr 6, 2017
Priority date	Apr 8, 2016
Publication date	Jan 15, 2019
Grant date	Jan 15, 2019

Abstract

A method for reconstructing a super-resolution image, including: 1) reducing the resolution of an original high-resolution image to obtain an equal low-resolution image, respectively expressed as matrix forms y h and y l ; 2) respectively conducting dictionary training on y l and y hl to obtain a low-resolution image dictionary D l ; 3) dividing the sparse representation coefficients α l and α hl into training sample coefficients α l _ train and α hl _ train and test sample coefficients α l _ test and α hl _ test ; 4) constructing an L-layer deep learning network using a root-mean-square error as a cost function; 5) iteratively optimizing network parameters so as to minimize the cost function by using the low-resolution image sparse coefficient α l _ train as the input of the deep learning network; 6) inputting the low-resolution image sparse coefficient α l _ test as the test portion into the trained deep learning network in 5), outputting to obtain a predicted difference image sparse coefficient {circumflex over (α)} hl _ test , computing an error between the {circumflex over (α)} hl _ test .

First claim

The invention claimed is:

1. A method for reconstructing a super-resolution image, comprising:
a) reducing the resolution of an original high-resolution image to obtain an equal low-resolution image, respectively expressed as matrix forms y h and y l , and computing a difference portion between two images, y hl =y h −y l ;
b) respectively conducting dictionary training on y l and y hl to obtain a low-resolution image dictionary D l , a difference image dictionary D hl and corresponding sparse representation coefficients α l and α hl ;
c) dividing the sparse representation coefficients α l and α hl into training sample coefficients α l _ train and α hl _ train and test sample coefficients α l _ test and α hl _ test ;
d) constructing an L-layer deep learning network using a root-mean-square error as a cost function;
e) iteratively optimizing network parameters so as to minimize the cost function by using the low-resolution image sparse coefficient α l _ train as the input of the deep learning network, using the corresponding difference image sparse coefficient α hl _ train as a target output and using {circumflex over (α)} hl _ train as a network-predicted difference image sparse coefficient, until a trained deep learning network is obtained;
f) inputting the low-resolution image sparse coefficient α l _ test as the test portion into the trained deep learning network in e), outputting to obtain a predicted difference image sparse coefficient {circumflex over (α)} hl _ test , computing an error between the {circumflex over (α)} hl _ test and a corresponding true difference image sparse coefficient α hl _ test , and verifying that the deep learning network obtained by training in e) is a mapping between the low-resolution image sparse coefficient and the difference image sparse coefficient when the error is less than a given threshold; and
g) expressing the low-resolution image to be subjected to resolution increase as the matrix form z l , expressing z l with the dictionary D l , recording a corresponding sparse coefficient as β l , inputting β l into the trained deep learning network to obtain a predicted difference image sparse coefficient β hl , reconstructing a difference portion {umlaut over (z)} hl with the dictionary D hl ; and finally reducing {umlaut over (z)} h ={umlaut over (z)} hl +z l into an image form to reconstruct a corresponding high-resolution image z h .

2. The method of claim 1 , wherein in a), firstly, a high-resolution image in a training sample library is cut into N d×dimage blocks; the resolution of each image block is reduced to obtain N corresponding low-resolution image blocks; then column vectors formed by stretching the high-resolution image blocks compose a matrix y h ∈R d 2 ×N , and column vectors formed by stretching the low-resolution image blocks compose a matrix y l ∈R d 2 ×N ; and the difference portion y hl =y h −y l of two matrixes is obtained through computation.

3. The method of claim 1 , wherein in b), dictionary training is respectively conducted on y l and y hl to obtain a corresponding low-resolution image dictionary D l , a difference image dictionary D hl and corresponding sparse representation coefficients α l and α hl , equivalent to solving optimization problems as follows:

min

D
l

,

α
l

⁢

||

α
l

⁢

||
0

⁢

⁢

subject
⁢

⁢
to

⁢

||

y
l

-

D
l

⁢

α
l

⁢

||
F
2

⁢

≤
ɛ

min

D
hl

,

α
hl

⁢

||

α
hl

⁢

||
0

⁢

⁢

subject
⁢

⁢
to

⁢

||

y
hl

-

D
hl

⁢

α
hl

⁢

||
F
2

⁢

≤
ɛ

wherein ε is a reconstruction error threshold.

4. The method of claim 1 , wherein in d), the constructed deep learning network comprises L layers; the output of each layer is recorded as x l , l=0, 1, 2, . . . , L, wherein x 0 is a network input and then the output of an l th layer is:
x l =f l ( W l x l-1 +b l ), l= 1,2, . . . , L

wherein W l and b l respectively indicate the weight and the bias term of the l th layer, f l (⋅) is an activation function of the l th layer, and the output of the l th layer is a network prediction.

5. The method of claim 1 , wherein in e), an implicit relationship between the low-resolution image sparse coefficient α l _ train and the difference image sparse coefficient α hl _ train is trained by the deep learning network, and by using the low-resolution image sparse coefficient α l _ train as the input of the deep learning network and using the difference image sparse coefficient α hl _ train as a supervision, the network-predicted difference image sparse coefficient is recorded as
{circumflex over (α)} hl _ train =f L ( . . . f l ( W l α l _ train +b 1 ))  (4)

wherein W l and b l respectively

Assignees

• Univ Wuhan

Inventors

• Shao Zhenfeng
• Wang Lei
• Wang Zhongyuan
• Cai Jiajun

Classifications

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• Y02T10/40

  Engine management systems · CPC title

• G06T2207/20224

  Image subtraction · CPC title

• G06K9/66 Primary

  Physics · mapped topic

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title