Light weight multi-branch and multi-scale person re-identification

Patent metadata

Field	Value
Publication number	US-12307802-B2
Application number	US-201917764100-A
Country	US
Kind code	B2
Filing date	Dec 20, 2019
Priority date	Dec 20, 2019
Publication date	May 20, 2025
Grant date	May 20, 2025

Abstract

A system for lightweight multi-branch and multi-scale (LMBMS) re-identification is described herein. The system includes a convolutional neural network trained for person identification, wherein the convolutional neural network comprises a series of residual blocks that obtain input from a head network of the convolutional neural network. The system also includes a plurality of refine blocks, wherein one or more refine blocks take as input features from a residual block of the series of residual blocks, wherein the features are at input at different scales and different resolutions and an output of the plurality of refine blocks is a plurality of features in a same feature space. A channel-wise attention mechanism may merge the plurality of features and generate final dynamic features.

First claim

What is claimed is:

1. A system comprising:
interface circuitry;
machine-readable instructions; and
at least one processor circuit to be programmed by the machine-readable instructions to:
extract, using a first residual block of a convolutional neural network, a first local feature of a subject from one or more images of the subject input to the convolutional neural network, the first local feature associated with a first resolution;
extract, using a second residual block of the convolutional neural network, a second local feature of the subject from the one or more images, the second local feature associated with a second resolution, the second resolution different than the first resolution;
process, using a first number of refine blocks, the first local feature to generate a refined first local feature, the first number of refine blocks selected based on the first resolution;
process, using a second number of refine blocks, the second local feature to generate a refined second local feature, the second number of refine blocks selected based on the second resolution, the second number of refine blocks different than the first number of refine blocks;
generate final dynamic features for the subject based on the refined first local feature and the refined second local feature;
identify the subject in a first image and a second image using the final dynamic features, the first image different than the second image, the first image and the second image different from the one or more images of the subject input to the convolutional neural network; and
output an indicator identifying the subject in the first image and an indicator identifying the subject in the second image.

2. The system of claim 1 , wherein the subject is a first subject, the one or more images includes the first subject and at least one other subject, and one or more of the at least one processor circuit is to:
derive isolated bounding boxes for each subject in the one or more images, wherein the isolated bounding boxes are input to the convolutional neural network.

3. The system of claim 1 , wherein one or more of the at least one processor circuit is to:
derive the first local feature from bounding boxes input to the first residual block,
derive the second local feature based on an output of the first residual block, the second local feature associated with a higher resolution than the first local feature.

4. The system of claim 1 , wherein the first number of refine blocks includes a first refine block and one or more of the at least one processor circuit is to cause the first refine block to perform a series of convolutions including a 1×1 convolution, a 3×3 convolution, and a second 1×1 convolution on the first local feature and multiply a result of the series of convolutions with a result of average pooling and another 1×1 convolution applied to the first local feature in a skip branch of the refine block.

5. The system of claim 1 , wherein one or more of the at least one processor circuit is to:
obtain a weight distribution for the refined first local feature and the refined second local feature according to a softmax function; and
generate the final dynamic features based on the weight distribution, the refined first local feature, and the refined second local feature.

6. A method comprising:
extracting, using a first residual block of a convolutional neural network and by at least one processor circuit programmed by at least one instruction, a first local feature of a subject from one or more images of the subject input to the convolutional neural network, the first local feature associated with a first resolution;
extracting, using a second residual block of the convolutional neural network, a second local feature of the subject from the one or more images, the second local feature associated with a second resolution, the second resolution different than the first resolution;
processing, using a first number of refine blocks, the first local feature to generate a refined first local feature, the first number of refine blocks selected based on the first resolution;
processing, using a second number of refine blocks, the second local feature to generate a refined second local feature, the second number of refine blocks selected based on the second resolution, the second number of refine blocks different than the first number of refine blocks;
generating final dynamic features for the subject based on the refined first local feature and the refined second local feature;
identifying the subject in a first image and a second image using the final dynamic features, the first image different than the second image, the first image and the second image different from the one or more images of the subject input to the convolutional neural network; and
outputting an indicator identifying the subject in the first image and an indicator identifying the subject in the second image.

7. The method of claim 6 , wherein the first image and the second image correspond to frames from a single camera view.

8. The method of claim 6 , wherein the first image is associated with a first camera view and the second image is associated with a second camera view, the first camera view different than the second camera view.

9. The method of claim 6 , wherein the subject is a first subject, the one or more images includes the first subject and at least one other subject, and further including:
deriving isolated bounding boxes for each subject in the one or more images, wherein the isolated bounding boxes are input to the convolutional neural network.

10. The method of claim 6 , further including:
deriving the first local feature from bounding boxes input to the first residual block; and
deriving the second local feature based on an output of the first residual block, the second local feature associated with a higher resolution than the first local feature.

11. The method of claim 6 , wherein the first number of refine blocks includes a first refine block and further including causing the first refine block to perform a series of convolutions including a 1×1 convolution, a 3×3 convolution, and a second 1×1 convolution on the first local feature and multiplying a result of this series of convolutions with a result of average pooling and another 1×1 convolution applied to the first local feature in a skip branch of the refine block.

12. The method of claim 6 , further including:
selecting the first number of refine blocks for the first local feature based on a scale of associated with the first local feature; and
selecting the second number of refine blocks for the second local feature based on a scale associated with the second local feature.

13. The method of claim 6 , further including:
obtaining a weight distribution for the refined first local feature and the refined second local feature according to a softmax function; and
generating the final dynamic features based on the weight distribution, the refined first local feature, and the refined second local feature.

14. The method of claim 6 , further including applying batch normalization to the final dynamic features.

15. The method of claim 6 , further including training the convolutional neural network using a triplet loss combined with a metric task and a cross entropy loss for classification.

16. At least one non-transitory machine-readable medium comprising machine-readable instructions to cause at least one processor circuit to at least:
extract, using a first residual block of a convolutional neural network, a first local feature of a subject from one or m

Assignees

• Intel Corp

Inventors

• Tao Hongzhi
• Li Qiang

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V10/82

  using neural networks · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title