Object re-identification using pose part based models

Patent metadata

Field	Value
Publication number	US-2022343639-A1
Application number	US-201917764093-A
Country	US
Kind code	A1
Filing date	Dec 6, 2019
Priority date	Dec 6, 2019
Publication date	Oct 27, 2022
Grant date	—

Abstract

An example apparatus for re-identifying objects includes an image receiver to receive a first image and a second image of an object with an identity. The apparatus also includes a fused model generator to fuse a global representation of the object with local representations of pose parts of the object to generate a fused representation of the object based on the first image. The apparatus further includes an object re-identifier to re-identify the object with the identity in the second image based on the fused representation.

First claim

1 . An apparatus for re-identifying objects in images, the apparatus comprising:
at least one memory; instructions; and processor circuitry to execute the instructions to at least:
receive a first image and a second image of an object with an identity;
fuse a global representation of the object with local representations of pose parts of the object to generate a fused representation of the object based on the first image; and
re-identify the object with the identity in the second image based on the fused representation.

2 . The apparatus of claim 1 , wherein the processor circuitry is to generate the global representation, the global representation including a feature map.

3 . The apparatus of claim 1 , wherein the processor circuitry is to estimate pose keypoints in the first image and generate a skeleton structure of the object based on the pose keypoints.

4 . The apparatus of claim 1 , wherein the processor circuitry is to generate the local representations of the pose parts based on a skeleton structure of the object and a feature map of the first image, the local representations including local part features.

5 . The apparatus of claim 1 , wherein the local representations include star structure models.

6 . The apparatus of claim 1 , a wherein the processor circuitry is to aggregate local part features using concatenation.

7 . The apparatus of claim 1 , wherein the processor circuitry is to aggregate local part features using a weighted summation of the local part features.

8 . The apparatus of claim 1 , wherein the processor circuitry is to extract the local representations from the global representation using regional average pooling.

9 . The apparatus of claim 1 , wherein the processor circuitry includes a deep neural network trained using a fused-triplet loss function.

10 . The apparatus of claim 1 , wherein the processor circuitry is to train a deep neural network trained to generate the fused representations and re-identify the object.

11 . A method for re-identifying objects in images, the method comprising:
receiving, via a processor, a first input object image and a second input object image of an object with an identity; globally modeling, via the processor, the object based on the first input object image to generate a global representation, the global representation including a feature map; estimating, via the processor, pose keypoints of the object in the first input object image; generating a skeleton structure of the object based on the pose keypoints; modeling, via the processor, local parts of the object in the first input object image based on the feature map and the pose keypoints to generate local representations; fusing, via the processor, the global representation of the object with the local representations of pose parts of the object to generate a fused representation of the object based on the first input object image; and re-identifying, via the processor, the object with the identity in the second input object image based on the fused representation.

12 . The method of claim 11 , further including aggregating local part features of the local representations using a concatenation of the local part features.

13 . The method of claim 11 , further including aggregating local part features of the local representations using a weighted summation of the local part features.

14 . The method of claim 11 , wherein modeling the local parts includes extracting the local representations from the global representation using regional average pooling.

15 . The method of claim 11 , wherein re-identifying the object includes receiving the second input object image at a trained deep neural network and outputting a re-identification of the object.

16 . The method of claim 11 , wherein globally modeling the object includes generating bounding boxes enclosing regions of an input object image corresponding to different pose parts of an object.

17 . The method of claim 11 , wherein estimating the pose keypoints includes estimating the pose keypoints using a number of pose keypoints based on a category of the object.

18 . The method of claim 11 , wherein fusing the global representation with the local representations includes training a deep neural network to perform a global transformation on aggregated local features using a triplet hard loss function.

19 . The method of claim 11 , further including individually training a plurality of deep neural networks to globally model the object, estimate the pose keypoints, model the local parts of the object, and fuse the global representation of the object with the local representations of the object.

20 . The method of claim 11 , further including simultaneously training an integrated deep neural network to globally model the object, estimate the pose keypoints, model the local parts of the object, and fuse the global representation of the object with the local representations of the object.

21 . A system for re-identifying objects in images, the system comprising:
means for receiving a first image and a second image of an object with an identity; means for fusing a global representation of the object with local representations of pose parts of the object to generate a fused representation of the object based on the first image; and means for re-identifying the object with the identity in the second image based on the fused representation.

22 . The system of claim 21 , further including means for generating the global representation, the global representation including a feature map.

23 . The system of claim 21 , further including means for estimating pose keypoints in the first image to generate a skeleton structure of the object.

24 . The system of claim 21 , further including means for generating the local representations of the pose parts based on a skeleton structure of the object and a feature map of the first image, the local representations including local part features.

25 . The system of claim 21 , wherein the local representations include star structure models.

Assignees

• Intel Corp

Inventors

• Li Jianguo
• Li Shuyuan
• Tang Hanlin

Classifications

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06V10/806 Primary

  of extracted features · CPC title

• G06V20/53

  Recognition of crowd images, e.g. recognition of crowd congestion · CPC title

• G06F18/253

  of extracted features · CPC title