Systems and methods for identifying users in media content based on poselets and neural networks

Patent metadata

Field	Value
Publication number	US-9704029-B2
Application number	US-201615284296-A
Country	US
Kind code	B2
Filing date	Oct 3, 2016
Priority date	Dec 17, 2014
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

Systems, methods, and non-transitory computer-readable media can receive a first image including a representation of a first user. A second image including a representation of a second user can be received. A first set of poselets associated with the first user can be detected in the first image. A second set of poselets associated with the second user can be detected in the second image. The first image including the first set of poselets can be inputted into a first instance of a neural network to generate a first multi-dimensional vector. The second image including the second set of poselets can be inputted into a second instance of the neural network to generate a second multi-dimensional vector. A first distance metric between the first multi-dimensional vector and the second multi-dimensional vector can be determined.

First claim

What is claimed is:

1. A computer-implemented method comprising:
determining, by a computing system, a first distance between a first value and a second value, wherein the first value is based on provision of a first image including a first set of poselets associated with a first user into a first instance of a neural network and the second value is based on provision of a second image including a second set of poselets associated with the first user into a second instance of the neural network;
determining, by the computing system, a second distance between the first value and a third value, wherein the third value is based on provision of a third image including a third set of poselets associated with a second user into a third instance of the neural network; and
training, by the computing system, the neural network to cause the first distance to be less than the second distance.

2. The computer-implemented method of claim 1 , wherein at least one of the first value, the second value, and the third value is a multi-dimensional vector.

3. The computer-implemented method of claim 1 , wherein the first image corresponds to a query image, wherein the second image corresponds to a positive image, wherein the third image corresponds to a negative image, wherein the first image, the second image, and the third image are included in a set of training images.

4. The computer-implemented method of claim 1 , wherein the training of the neural network includes modifying one or more weights associated with the neural network via one or more neural network back-propagation processes.

5. The computer-implemented method of claim 1 , wherein the training of the neural network to cause the first distance to be lesser than the second distance is based on minimizing a loss metric, and wherein the loss metric is determined by calculating a maximum value between zero and (1−the second distance+the first distance).

6. The computer-implemented method of claim 5 , wherein the first distance corresponds to a first Euclidean distance between the first value and the second value, and wherein the second distance corresponds to a second Euclidean distance between the first value and the third value.

7. The computer-implemented method of claim 1 , further comprising:
determining that the first distance is less than a specified distance threshold.

8. The computer-implemented method of claim 1 , wherein the first set of poselets and the second set of poselets are included within a set of defined poselets.

9. The computer-implemented method of claim 8 , wherein the set of defined poselets is associated with at least one of a body portion, a combination of multiple body portions, a front facial portion, a side facial portion, a head portion, a hair portion, a wearable article portion, a perspective, or a pose.

10. The computer-implemented method of claim 1 , wherein each of the first value and the second value is associated with 256 dimensions.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
determining a first distance between a first value and a secondvalue, wherein the first value is based on provision of a first image including a first set of poselets associated with a first user into a first instance of a neural network and the second value is based on provision of a second image including a second set of poselets associated with the first user into a second instance of the neural network;
determining a second distance between the first value and a third value, wherein the third value is based on provision of a third image including a third set of poselets associated with a second user into a third instance of the neural network; and
training the neural network to cause the first distance to be less than the second distance.

12. The computer-implemented method of claim 11 , wherein at least one of the first value, the second value, and the third value is a multi-dimensional vector.

13. The system of claim 11 , wherein the training of the neural network to cause the first distance to be lesser than the second distance is based on minimizing a loss metric, and wherein the loss metric is determined by calculating a maximum value between zero and (1−the second distance+the first distance).

14. The system of claim 13 , wherein the first distance corresponds to a first Euclidean distance between the first value and the second value, and wherein the second distance corresponds to a second Euclidean distance between the first value and the third value.

15. The system of claim 11 , wherein the instructions cause the system to further perform:
determining that the first distance is less than a specified distance threshold.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
determining a first distance between a first value and a second value, wherein the first value is based on provision of a first image including a first set of poselets associated with a first user into a first instance of a neural network and the second value is based on provision of a second image including a second set of poselets associated with the first user into a second instance of the neural network;
determining a second distance between the first value and a third value, wherein the third value is based on provision of a third image including a third set of poselets associated with a second user into a third instance of the neural network; and
training the neural network to cause the first distance to be less than the second distance.

17. The computer-implemented method of claim 16 , wherein at least one of the first value, the second value, and the third value is a multi-dimensional vector.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the training of the neural network to cause the first distance to be lesser than the second distance is based on minimizing a loss metric, and wherein the loss metric is determined by calculating a maximum value between zero and (1−the second distance+the first distance).

19. The non-transitory computer-readable storage medium of claim 18 , wherein the first distance corresponds to a first Euclidean distance between the first value and the second value, and wherein the second distance corresponds to a second Euclidean distance between the first value and the third value.

20. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions cause the system to further perform:
determining that the first distance is less than a specified distance threshold.

Assignees

• Facebook Inc

Inventors

• Bourdev Lubomir
• Zhang Ning
• Paluri Balamanohar
• Taigman Yaniv
• Fergus Robert D

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/24133

  Distances to prototypes · CPC title