Image retrieval method based on variable-length deep hash learning

Patent metadata

Field	Value
Publication number	US-10776685-B2
Application number	US-201815990586-A
Country	US
Kind code	B2
Filing date	May 25, 2018
Priority date	Dec 3, 2015
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

This invention is an image retrieval method based on bit-scalable deep hashing learning. According to the method, the training images is used to generate a batch of image triples, wherein each of the triples contains two images with the same label and one image with a different label. The purpose of model training is to maximize a margin between matched image pairs and unmatched image pairs in the Hamming space. The deep convolutional neural network is utilized to train the model in an end-to-end fasion, where discriminative images features and has functions are simultaneously optimized. Furthermore, each bit of the hashing codes is unequally weighted so that we can manipulate the code length by truncating the insignificant bits. It is also shown that the generated bit-scalable hashing codes well preserve the discriminative powers with sorter code lengths.

First claim

What is claimed is:

1. An image retrieval method based on bit-scalable deep hashing learning, comprising:
S1, reorganizing the training images into a batch of image triples;
S2, feeding the image triples into the deep convolutional neural network;
S3, calculating the loss from each image, and training the deep convolutional neural network via a back propagation algorithm;
S4, truncating the unimportant hash bit according to the input of a user, and calculating the weighted Hamming distance between a query image and each image in the database;
S5, ranking images in the database in a descending order according to the weighted Hamming distance in S4, the ranking result is the returned similarity retrieval result;
wherein the deep convolutional neural network in S2 comprises several convolution layers and pooling layers, part of fully-connected layers, a sign-like tangent function layer, and a hashing weight layer; The sign-like tangent function layer is a sign-like tangent function with a feature vector as the input and each dimension ranging in [−1, 1], where the function has a harmonic parameter used to control its smoothness, and as the parameter is smaller, the function is smoother; And the hashing weight layer is a deep network layer with the output of the sign-like tangent function layer as its input, each dimension corresponding to a weight.

2. The image retrieval method based on bit-scalable deep hashing learning according to claim 1 , wherein the image triple in S1 specifically comprises two images with the same label and one image with a different label.

3. The image retrieval method based on bit-scalable deep hashing learning according to claim 1 , wherein a training process of the deep convolutional neural network in S3 is an end-to-end process, thereby achieving joint optimization of an image feature learning and a hashing function learning;
the image feature is the input of the penultimate fully-connected layer of the deep convolutional network;
and the parameters of the hashing function are directly corresponding to all parameters contained by the last fully-connected layer.

4. The image retrieval method based on bit-scalable deep hashing learning according to claim 1 , wherein a loss of each image in S3 is calculated by maximizing a margin between matched image pairs and unmatched image pairs in the Hamming space, satisfying:

W
*

=

min
W

⁢

∑
i

⁢

max
⁢

{

H
⁡

(

h
i

,

h
i
+

)

-

H
⁡

(

h
i

,

h
i
-

)

,
C

}

where W is a parameter of a deep convolutional neural network, H(⋅, ⋅) represents a distance between two hash codes in a Hamming space, and C is a constant for reducing influence of noise to the model.

5. The image retrieval method based on bit-scalable deep hashing learning according to claim 1 , wherein the unimportant hash bit in S4 is determined by the weight of the hash bit, where the smaller weight represents the less importance of the hash bit.

6. The image retrieval method based on bit-scalable deep hashing learning according to claim 1 , wherein calculating the weighted Hamming distance in S4 comprises the following steps:
S-a, acquiring the weight of an important hash bit;
S-b, constructing a look-up table having a length of 2 l , where l is the length of an important hash bit, and a value 2 l is equal to all possible XOR results generated by two hash codes;
S-c, calculating a weighted Hamming affine distance under each XOR result, and storing the result at a corresponding position of the look-up table; and
S-d, calculating XOR values of the two hash codes, and returning corresponding values thereof in the look-up table.

7. The image retrieval method based on bit-scalable deep hashing learning according to claim 6 , wherein in the case of a larger l, the look-up table having a length of 2 l may be split into a plurality of sub-tables having equal lengths, each sub-table corresponding to a hash code having a fixed length; each value in the sub-tables represents weighted similarity of a corresponding sub-hash code; and a weighted affine distance between two hash codes may be finally obtained by accumulating weighted affinities of each hash code.

Assignees

• Univ Sun Yat Sen

Inventors

• Lin Liang
• Zhang Ruimao
• Wang Qing
• Jiang Bo

Classifications

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06N3/045

  Combinations of networks · CPC title