Pedestrian re-identification method based on spatio-temporal joint model of residual attention mechanism and device thereof

Patent metadata

Field	Value
Publication number	US-11468697-B2
Application number	US-202017121698-A
Country	US
Kind code	B2
Filing date	Dec 14, 2020
Priority date	Dec 31, 2019
Publication date	Oct 11, 2022
Grant date	Oct 11, 2022

Abstract

The disclosure provides a pedestrian re-identification method based on a spatio-temporal joint model of a residual attention mechanism and a device thereof. The method includes: performing feature extraction for an input pedestrian with a pre-trained ResNet-50 model; constructing a residual attention mechanism network including a residual attention mechanism module, a feature sampling layer, a global average pooling layer and a local feature connection layer; calculating a feature distance by using a cosine distance and denoting the feature distance as a visual probability according to the trained residual attention mechanism network; performing modeling for a spatio-temporal probability according to camera ID and frame number information in a pedestrian tag of a training sample, and performing Laplace smoothing for a probability model; and calculating a final spatio-temporal joint probability by using the visual probability and the spatio-temporal probability to obtain a pedestrian re-identification result.

First claim

What is claimed is:

1. A method, comprising:
a) performing feature extraction for an input pedestrian x with a ResNet-50 model obtained through pre-training to obtain a feature matrix denoted as f;
b) constructing a residual attention mechanism network with a network structure comprising a residual attention mechanism module, a feature sampling layer, a global average pooling layer and a local feature connection layer;
c) taking the feature matrix f with dimensions being H×W×C obtained in a) as an input of the residual attention mechanism network, and taking corresponding identity information y as a target output, wherein H, W, C refer to a length, a width and a channel number of a feature map, respectively;
performing channel averaging for each spatial position of the feature matrix f as a spatial weight matrix according to the residual attention mechanism module; activating the spatial weight matrix by softmax to ensure that a convolution kernel learns different features, and calculating an attention mechanism map M SA to obtain a feature matrix F RSA with dimensions being H×W×C by F RSA =f*M SA +f;
d) sampling the feature matrix F RSA with dimensions being H×W×C into local feature matrixes (F RSA 1 , F RSA 2 . . . , F RSA 6 ) with dimensions being

H
6

×
W
×
C

by the feature sampling layer, and calculating local feature vectors (V RSA 1 , V RSA 2 . . . V RSA 6 ) by the global average pooling layer;
e) connecting local features into a feature vector V RSA by the local feature connection layer, and calculating a cross entropy loss between the feature vector V RSA and the pedestrian identity y to obtain the trained residual attention mechanism network after training;
f) obtaining feature vectors V RSA-α and V RSA-β corresponding to tested pedestrian images x α and x β respectively according to the trained residual attention mechanism network obtained in e), and calculating a feature distance based on a cosine distance and denoting the feature distance as a visual probability P V ;
g) performing modeling for a spatio-temporal probability according to camera ID and frame number information in a pedestrian tag of a training sample, and calculating the spatio-temporal probability P ST according to the obtained spatio-temporal model; and
h) calculating a final joint spatio-temporal probability using the visual probability P V obtained in f) and the spatio-temporal probability P ST obtained in g) to obtain a pedestrian re-identification result.

2. The method of claim 1 , wherein in c), the residual attention mechanism model is defined as follows:

Q
⁡

(

i
,
j

)

=

∑

t
=
0

C

⁢

⁢

f
t

⁡

(

i
,
j

)

C

M
SA

⁡

(

i
,
j

)

=

e

Q
⁡

(

i
,
j

)

Σ
⁡

(

i
,
j

)

e

Q
⁡

(

i
,
j

)

F

RSA
t

⁡

(

i
,
j

)

=

f
t

⁡

(

i
,
j

)

⁢

M
SA

⁡

(

i
,
j

)

+

f
t

⁡

(

i
,
j

)

,

wherein (i,j) refers to spatial position information, t refers to a channel serial number, f t (i,j) refers to a pixel point with the spatial position being (i,j) in a t-th channel of the feature matrix f, e refers to a base of a natural logarithm, and F RSA (i,j) refers to a pixel point with the spatial position being (i,j) in the feature matrix F RSA .

3. The method of claim 1 , wh

Assignees

• Univ Wuhan

Inventors

• Shao Zhenfeng
• Wang Jiaming

Classifications

• G06V40/23 Primary

  Recognition of whole body movements, e.g. for sport training · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title