Vector-based face recognition algorithm and image search system

Patent metadata

Field	Value
Publication number	US-2016335482-A1
Application number	US-201615223446-A
Country	US
Kind code	A1
Filing date	Jul 29, 2016
Priority date	Nov 7, 2014
Publication date	Nov 17, 2016
Grant date	—

Abstract

Systems and methods for performing face recognition and image searching are provided. A system for face recognition and image searching includes an ingestion system, a search system, a user device, and a database of galley files that include feature vectors. The ingestion system crawls the internet starting with a seed URL to scrape image files and generate feature vectors. Feature vectors of images input by a user may be compared by the search system to feature vectors in the gallery files. A method for generating feature vectors includes landmark detection, component aligning, texture mapping, vector computation, comparing cluster centers defined by vectors stored in a database with vectors generated based on an input image, linear discriminant analysis, and principal component analysis.

First claim

1 .- 16 . (canceled)

17 . A method, performed by a processor, for creating a feature vector representing an image of a face, the method comprising:
receiving an image of a face; computing a first vector based on a portion of the image of the face; creating a second vector based on a similarity of the first vector to each of a predetermined number of a first plurality of vectors; generating a feature vector by concatenating the second vector with a second plurality of vectors; and storing the feature vector, along with associated metadata as part of a gallery file, in a database.

18 . The method of claim 17 , comprising:
prior to computing the first vector based on the portion of the image of the face: detecting landmarks on the image of the face and associating the detected landmarks with points; subjecting the points to a transformation; aligning a portion of the image of the face in accordance with the transformation; and performing texture mapping on the portion of the image of the face.

19 . The method of claim 17 , wherein the second vector and each of the second plurality of vectors each correspond to a respective portion of the image of the face

20 . The method of claim 17 , wherein creating the second vector includes:
calculating a distance of the first vector from each of the first plurality of vectors, and selecting from the calculated distances, for entry into the second vector, the predetermined number of calculated distances that are smallest.

21 . The method of claim 20 , wherein each of the first plurality of vectors defines a center of a cluster, each cluster comprising a third plurality of vectors computed based on a portion of a different image of a face.

22 . The method of claim 21 , wherein each of the first plurality of vectors is calculated using a radial basis function.

23 . The method of claim 21 , wherein computing the first vector and computing each of the third plurality of vectors include computing local binary patterns.

24 . The method of claim 21 , wherein computing the first vector and computing each of the third plurality of vectors include computing a histogram oriented gradient.

25 . The method of claim 17 , comprising:
subjecting the feature vector to linear discriminant analysis.

26 . The method of claim 17 , comprising:
subjecting the feature vector to principal component analysis.

27 . The method of claim 17 , wherein the portion of the image of the face is a component image representing a part of the face selected from the group consisting of: eyes, eyebrows, nose, mouth, and entire face.

28 . The method of claim 17 , wherein the portion of the image of the face is a rectangular sub-portion of a component image representing a part of the face selected from the group consisting of: eyes, eyebrows, nose, mouth, and entire face.

29 . The method of claim 17 , wherein receiving an image of a face comprises:
receiving, by a crawler, input of a seed network address, accessing, by the crawler, the seed network address, and retrieving, by the crawler, an image located at the seed network address.

30 . The method of claim 29 , wherein receiving an image of a face further comprises:
detecting, by the crawler, on a page located at the seed network address, a second network address, accessing, by the crawler, the second network address, and retrieving, by the crawler, an image located at the second network address.

31 . The method of claim 29 , wherein receiving an image of a face further comprises:
determining, by a duplicate filter, whether the image has previously been retrieved, and if the image has previously been retrieved, preventing creation of the a new feature vector corresponding to the image.

32 . The method of claim 17 , comprising:
associating the feature vector with an identity tag; generating a covariance matrix based on the feature vector.

33 . A system for creating a searchable database of feature vectors representing images of faces, comprising:
an enrollment server configured to: receive an image of a face; compute a first vector based on a portion of the image of the face, create a second vector based on a similarity of the first vector to each of a predetermined number of a first plurality of vectors, and generate a feature vector by concatenating the second vector with a second plurality of vectors; and a database configured to store the feature vector, along with associated metadata as part of a gallery file.

34 . The system of claim 33 , comprising:
a crawler configured to retrieve the image of the face.

35 . The system of claim 33 , comprising:
a duplicate filter configured to prevent creation of a new feature vector if the image has previously been retrieved.

36 . The system of claim 33 , wherein the enrollment server is configured to:
prior to computing the first vector based on the portion of the image of the face: detect landmarks on the image of the face and associating the detected landmarks with points; subject the points to a transformation; align a portion of the image of the face in accordance with the transformation; and perform texture mapping on the portion of the image of the face.

37 . The system of claim 33 , wherein the second vector and each of the second plurality of vectors each correspond to a respective portion of the image of the face.

38 . A method, performed by a search system, for searching a database of feature vectors representing images of faces to select resulting images of faces that are similar to an input image of a face, comprising:
receiving an image of a face; computing a first vector based on a portion of the image of the face; creating a second vector based on a similarity of the first vector to each of a predetermined number of a first plurality of vectors; generating a query feature vector by concatenating the second vector with a second plurality of vectors; selecting a plurality of resulting images of faces based on a comparison of the query feature vector with a plurality of feature vectors stored in gallery files in a database.

39 . The method of claim 38 , comprising:
prior to computing the first vector based on the portion of the image of the face: detecting landmarks on the image of the face and associating the detected landmarks with points; subjecting the points to a transformation; aligning a portion of the image of the face in accordance with the transformation; and performing texture mapping on the portion of the image of the face.

40 . The method of claim 38 , wherein the second vector and each of the second plurality of vectors each correspond to a respective portion of the image of the face

41 . The method of claim 38 , wherein selecting a plurality of resulting images comprises:
comparing the query feature vector to each of the plurality of feature vectors stored in gallery files, including calculating a distance between the query feature vector and each of the plurality of feature vectors stored in gallery files, and assigning each of the plurality of feature vectors stored in the gallery files a normalized similarity score based on the calculated distance of the respective feature vector to the query feature vector.

42 . The method of claim 38 , wherein selecting a plurality of resulting images comprises:
transmitting a query template from a se

Assignees

• Noblis Inc

Inventors

• Klare Brendan
• Klontz Joshua
• Blanton Austin

Classifications

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V40/171 Primary

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title