Method, apparatus, and electronic device for training place recognition model

Patent metadata

Field	Value
Publication number	US-12100192-B2
Application number	US-202117374810-A
Country	US
Kind code	B2
Filing date	Jul 13, 2021
Priority date	May 10, 2019
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

A computer device extracts local features of sample images based on a first part of a convolutional neural network (CNN) model. The sample images comprise a plurality of images taken at the same place. The device; aggregates the local features into feature vectors having a first dimensionality based on a second part of the CNN model. The device obtains compressed representation vectors of the feature vectors based on a third part of the CNN model. The compressed representation vectors have a second dimensionality less than the first dimensionality. The device trains the CNN model, and obtains a trained CNN mode satisfying a preset condition in accordance with the training.

First claim

What is claimed is:

1. A method for training neural networks, performed by a computer device, the method comprising:
extracting local features of sample images based on a first part of a convolutional neural network (CNN) model, the sample images comprising a first image, a plurality of second images taken at the same place as the first image, and a plurality of third images taken at a different place from the first image;
aggregating the local features into feature vectors having a first dimensionality based on a second part of the CNN model, the feature vectors including a first feature vector corresponding to the first image, second feature vectors corresponding to the second images, and third feature vectors corresponding to the third images;
obtaining compressed representation vectors of the feature vectors based on a third part of the CNN model, the compressed representation vectors having a second dimensionality less than the first dimensionality; and
training the CNN model, including adjusting model parameters of the first, second, and third parts of the CNN model based on the compressed representation vectors, the adjusting including:
constructing a loss function of the CNN model based on first distances and second distances, the first distances being distances between the first feature vector and the second feature vectors, and the second distances being distances between the first feature vector and the third feature vectors; and
back propagating the loss function through the CNN model to update the model parameters until the updated CNN model satisfies a preset convergence condition.

2. The method according to claim 1 , wherein training the CNN model comprises minimizing respective distances between at least a subset of the compressed representation vectors corresponding to the second plurality of images taken at the same place as the first image.

3. The method according to claim 1 , wherein obtaining the compressed representation vectors further comprises:
projecting the feature vectors onto a unit orthogonal space based on the third part to obtain the compressed representation vectors.

4. The method according to claim 3 , wherein the third part is a fully connected layer in the CNN model that receives the feature vectors, the fully connected layer comprising a quantity of neurons, the quantity being equal to the second dimensionality, a weight matrix of each of the neurons being a unit vector and having the first dimensionality, the weight matrices of the neurons satisfying an orthogonal relationship.

5. The method according to claim 4 , wherein adjusting the model parameters of the first part, the second part, and the third part comprises:
constructing a loss function of the CNN model based on an orthogonal constraint term of the weight matrices, the orthogonal constraint term being obtained based on the weight matrices of the neurons and a known unit vector.

6. The method according to claim 1 , wherein constructing the loss function comprises constructing the loss function as

L
=

∑
j

⁢

l
⁡

(

min
i

⁢

⁢

d
2

⁡

(

q
,

p
i
q

)

+
m
-

d
2

⁡

(

q
,

n
j
q

)

)

wherein L is the loss function, l is a maximum boundary loss, q is the first feature vector, p i q is an ith one of the second feature vectors, n j q is a j th one of the third feature vectors, m is a boundary constant, d represents vector distance calculation, and min represents minimum value calculation.

7. The method according to claim 1 , wherein extracting the local features of sample images comprises:
extracting the local features of the sample images by using a visual geometry group network (VGGNet) structure.

8. The method according to claim 1 , wherein aggregating the local features into feature vectors comprises:
aggregating the local features into the feature vectors by using a vector of locally aggregated descriptors network (NetVLAD) structure.

9. A computer device, comprising:
one or more processors; and
memory storing one or more programs, that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
extracting local features of sample images based on a first part of a convolutional neural network (CNN) model, the sample images comprising a first image, a plurality of second images taken at the same place as the first image, and a plurality of third images taken at a different place from the first image;
aggregating the local features into feature vectors having a first dimensionality based on a second part of the CNN model, the feature vectors including a first feature vector corresponding to the first image, second feature vectors corresponding to the second images, and third feature vectors corresponding to the third images;
obtaining compressed representation vectors of the feature vectors based on a third part of the CNN model, the compressed representation vectors having a second dimensionality less than the first dimensionality;
training the CNN model, including adjusting model parameters of the first, second, and third parts of the CNN model based on the compressed representation vectors, the adjusting including:
constructing a loss function of the CNN model based on first distances and second distances, the first distances being distances between the first feature vector and the second feature vectors, and the second distances being distances between the first feature vector and the third feature vectors; and
back propagating the loss function through the CNN model to update the model parameters until the updated CNN model satisfies a preset convergence condition.

10. The computer device according to claim 9 , wherein training the CNN model comprises m

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Bai Dongdong
• Ling Yonggen
• Liu Wei

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/7747

  Organisation of the process, e.g. bagging or boosting · CPC title