Long-tail large scale face recognition by non-linear feature level domain adaptation

Patent metadata

Field	Value
Publication number	US-10796135-B2
Application number	US-201816145608-A
Country	US
Kind code	B2
Filing date	Sep 28, 2018
Priority date	Sep 28, 2017
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

A computer-implemented method, system, and computer program product are provided for facial recognition. The method includes receiving, by a processor device, a plurality of images. The method also includes extracting, by the processor device with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors for each of the plurality of images. The method additionally includes generating, by the processor device with a feature generator, discriminative feature vectors for each of the feature vectors. The method further includes classifying, by the processor device utilizing a fully connected classifier, an identity from the discriminative feature vector. The method also includes control an operation of a processor-based machine to react in accordance with the identity.

First claim

What is claimed is:

1. A point of sale system with facial recognition, the point of sale system comprising:
one or more cameras;
a processor device and memory coupled to the processor device, the processing system programmed to:
receive a plurality of images from the one or more cameras;
extract, with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors from each of the plurality of images;
generate, with a feature generator, discriminative feature vectors for each of the feature vectors;
classify, with a fully connected classifier, an identity from the discriminative feature vectors; and
control an operation of the point of sale system to react in accordance with the identity.

2. The point of sale system as recited in claim 1 , further includes a communication system.

3. The point of sale system as recited in claim 1 , wherein the operation logs a customer into the point of sale system and greets the customer.

4. The point of sale system as recited in claim 1 , wherein the operation logs an employee into the point of sale system and greets the employee.

5. The point of sale system as recited in claim 1 , wherein the operation recognizes a customer and permits a purchase without an employee intervention.

6. The point of sale system as recited in claim 1 , wherein the one or more cameras is a ceiling mounted security camera.

7. The point of sale system as recited in claim 1 , further programmed to train the feature extractor, the feature generator, and the fully connected classifier with an alternative bi-stage strategy.

8. The point of sale system as recited in claim 1 , wherein the feature extractor shares covariance matrices across all classes to transfer intra-class variance from regular classes to the long-tail classes.

9. The point of sale system as recited in claim 1 , wherein the feature generator optimizes a softmax loss by joint regularization of weights and features through a magnitude of an inner product of the weights and features.

10. The point of sale system as recited in claim 1 , wherein the feature extractor averages the feature vector with a flipped feature vector, the flipped feature vector being generated from a horizontally flipped frame from one of the plurality of images.

11. The point of sale system as recited in claim 1 , wherein each of the plurality of images is selected from the group consisting of an image, a video, and a frame from the video.

12. The point of sale system as recited in claim 2 , wherein the communication system connects to a remote server that includes a facial recognition network.

13. The point of sale system as recited in claim 7 , wherein one stage of the alternative bi-stage strategy fixes the feature extractor and applies the feature generator to generate new transferred features that are more diverse and violate a decision boundary.

14. The point of sale system as recited in claim 7 , wherein one stage of the alternative bi-stage strategy fixes the fully connected classifier and updates the feature extractor and the feature generator.

15. A computer program product for a point of sale system with facial recognition, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
receiving, by a processor device, a plurality of images;
extracting, by the processor device with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors for each of the plurality of images;
generating, by the processor device with a feature generator, discriminative feature vectors for each of the feature vectors;
classifying, by the processor device utilizing a fully connected classifier, an identity from the discriminative feature vector; and
controlling an operation of the point of sale system to react in accordance with the identity.

16. A computer-implemented method for facial recognition in a point of sale system, the method comprising:
receiving, by a processor device, a plurality of images;
extracting, by the processor device with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors for each of the plurality of images;
generating, by the processor device with a feature generator, discriminative feature vectors for each of the feature vectors;
classifying, by the processor device utilizing a fully connected classifier, an identity from the discriminative feature vector; and
controlling an operation of the point of sale system to react in accordance with the identity.

17. The computer-implemented method as recited in claim 16 , wherein controlling includes recognizing a customer and permitting a purchase without an employee intervention.

18. The computer-implemented method as recited in claim 16 , wherein controlling includes logging a customer into the point of sale system and greeting the customer.

19. The computer-implemented method as recited in claim 16 , wherein controlling includes logging an employee into the point of sale system and greeting the employee.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Yu Xiang
• Yin Xi
• Sohn Kihyuk
• Chandraker Manmohan

Classifications

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title