Method and device for super-resolution image reconstruction based on dictionary matching

Patent metadata

Field	Value
Publication number	US-10339633-B2
Application number	US-201515749554-A
Country	US
Kind code	B2
Filing date	Nov 4, 2015
Priority date	Nov 4, 2015
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

The present application provides a method and a device for super-resolution image reconstruction based on dictionary matching. The method includes: establishing a matching dictionary library; inputting an image to be reconstructed into a multi-layer linear filter network; extracting a local characteristic of the image to be reconstructed; searching the matching dictionary library for a local characteristic of a low-resolution image block having the highest similarity with the local characteristic of the image to be reconstructed; searching the matching dictionary library for a residual of a combined sample where the local characteristic of the low-resolution image block with the highest similarity is located; performing interpolation amplification on the local characteristic of the low-resolution image block having the highest similarity; and adding the residual to a result of the interpolation amplification to obtain a reconstructed high-resolution image block. The local characteristics of the image to be reconstructed extracted by the multi-layer linear filter network have higher precision. Thus, a higher matching degree can be obtained during subsequent matching with the matching dictionary library, and the reconstructed image has a better quality. Therefore, the present invention can greatly improve the quality of the high-resolution image to be reconstructed.

First claim

What is claimed is:

1. A method of super-resolution image reconstruction based on dictionary matching, wherein comprising:
establishing a matching dictionary library;
inputting an image block to be reconstructed into a multi-layer linear filter network, and extracting a local characteristic of the image to be reconstructed;
searching the matching dictionary library for a local characteristic of a low-resolution image block having highest similarity with the local characteristic of the image block to be reconstructed, wherein the step for extracting a local characteristic of the image comprises:
Step 1 : wherein a multi-layer linear filter network comprises a filter layer, filtering an input image block to be reconstructed by a first-stage filter of the filter layer using N linear filter windows with different sizes to obtain corresponding N filtered images and output to the next stage filter, wherein the filtered image includes a line characteristic of the image, where N is an integer greater than one;
Step 2 : filtering the N filtered images output from the first-stage filter by a second-stage filter of the filter layer using M linear filter windows with different sizes to obtain corresponding M×N filtered images, where M is an integer greater than one;
Step 3 : outputting all the filtered images obtained by each stage filter to a next stage filter repeatedly; filtering all filtered images output from the previous stage filter by the next stage filter using multiple linear filter windows with different sizes until filtering by the last stage filter is completed; outputting all filtered images to the mapping layer of the multi-layer linear filter network;
Step 4 : performing binarization on all the filtered images of the filter layer by the mapping layer to output to the output layer of the multi-layer linear filter network; and
Step 5 : if the input image of the multi-layer linear filter network is a local image block, concatenating and outputting, by the output layer, the binarized filtered image output by the mapping layer to obtain a local characteristic of the image; if the input image of the multi-layer linear filter network is a whole image, making, by the output layer, a block histogram for each binarized filtered image output by the mapping layer, and then performing convergence for output to obtain a local characteristic of the image;

searching the matching dictionary library for a residual of a combined sample in which the local characteristic of the low-resolution image block having the highest similarity is located; and
performing interpolation amplification on the local characteristic of the low-resolution image block having the highest similarity, and adding the residual to a result of the interpolation amplification to obtain a reconstructed high-resolution image block.

2. The method of claim 1 , wherein the step of establishing a matching dictionary library comprises:
acquiring a plurality of high-resolution image blocks; down-sampling the plurality of high-resolution image blocks respectively to obtain low-resolution image blocks respectively corresponding to the high-resolution image blocks; and forming a pair of training samples using a high-resolution image block and the corresponding low-resolution image block;
subtracting the low-resolution image block in each pair of training samples from the high-resolution image block after the step of interpolation amplification to obtain a residual for the pair of training samples;
inputting the low-resolution image block of each pair of training samples into the multi-layer linear filter network to extract the local characteristic of the low-resolution image block of each pair of training samples;
splicing the local characteristic of the low-resolution image block of the each pair of training samples and the residual for the pair of training samples together to form the combined sample of the training samples; and
using K-mean clustering to train multiple combined samples to obtain a matching dictionary library.

3. The method of claim 1 , wherein the steps of filtering comprise:
for each filter window with a preset size, performing linear filtering of pixel (i 0 ,j 0 ) of the image in middle of the filter window by using linear filters in different directions using a response formula as follows:

f
⁡

(

P
F

)

=

|

min
(

∑

i
,

j
∈

L
k

⁢

L
k

)

|

⁢
k

=
1

,
2
,

L
⁢

⁢
N

(
1
)

where P F is the local image block of size of the filter windows, L k is a local line in direction k,(k =1,2, . . . N) in the filter window, (i,j) is coordinates of a pixel point on line L k , wherein line L k is defined as follows:
L k ={( i,j ): j=S k ( i−i 0 )+ j 0 ,i ∈P F }  (2)

where S k is the slope of line L k .

4. The method of claim 3 , wherein Step 4 comprises:
performing binarization on the filtered images using:

LB
⁡

(
x
)

=

{

1
,

if
⁢

Assignees

• Univ Peking Shenzhen Graduate School

Inventors

• Zhao Yang
• Wang Ronggang
• Gao Wen
• Wang Zhenyu
• Wang Wenmin

Classifications

• G06V10/772

  Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2414

  Smoothing the distance, e.g. radial basis function networks [RBFN] · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title