Facial recognition for multi-stream video using high probability group

Patent metadata

Field	Value
Publication number	US-11334746-B2
Application number	US-201916400293-A
Country	US
Kind code	B2
Filing date	May 1, 2019
Priority date	May 1, 2019
Publication date	May 17, 2022
Grant date	May 17, 2022

Abstract

Techniques are provided for facial recognition using a high probability group database. One method comprises maintaining (i) a first database of facial images of individuals, and (ii) a second database of facial images comprising a subset of the individuals from the first database based on a probability of individuals appearing in sequences of image frames at a given time; applying a face detection algorithm to sequences of image frames to identify one or more faces in the sequences of images; and applying a facial recognition to at least one sequence of image frames using at least the second database to identify one or more individuals in the at least one sequence of image frames. The second database is comprised of facial images of: (i) individuals from multiple angles; (ii) individuals that appeared in prior image frames; and/or (iii) individuals that appeared in an image frame generated by a plurality of cameras.

First claim

What is claimed is:

1. A method, comprising:
maintaining a first database of facial images of a plurality of individuals;
maintaining a second database of facial images comprising a subset of the individuals from the first database of images, wherein the subset is obtained based on a probability of individuals appearing in one or more sequences of image frames at a given time, wherein the first data database and the second database are maintained on separate computing devices;
obtaining the one or more sequences of image frames generated by one or more cameras;
applying a face detection algorithm to at least one of the sequences of image frames to identify one or more faces in the one or more sequences of image frames;
applying facial recognition to at least one sequence of image frames using at least the second database to identify one or more individuals in the at least one sequence of image frames; and
in response to determining that the identified one or more faces comprises at least one new face that does not satisfy a predefined similarity metric with a given face in the second database: (i) adding a new identifier for the at least one new face to the second database and at least one new facial image of the at least one new face to the second database, (ii) tracking, based at least in part on the new identifier and the at least one new facial image, the at least one new face to obtain one or more additional facial images of the at least one new face, and (iii) in response to obtaining a specified number of facial images of the at least one new face, matching the facial images of the at least one new face to images of known faces in the first database,
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 , wherein the one or more sequences of image frames are obtained from a plurality of streams of video images.

3. The method of claim 1 , wherein the second database of facial images is stored in one or more of a local memory, a cache, an edge device, a cloud device and an Internet device.

4. The method of claim 1 , wherein the second database of facial images comprises a plurality of images of one or more individuals from a plurality of angles.

5. The method of claim 1 , wherein the second database is comprised of facial images of one or more individuals that appeared in one or more prior image frames.

6. The method of claim 1 , wherein the second database is comprised of facial images of one or more individuals that appeared in at least one image frame generated by a plurality of cameras.

7. The method of claim 1 , wherein the second database is generated using facial tracking to track a face over time by comparing regions of a given image frame comprising a facial image to corresponding regions of one or more prior image frames.

8. The method of claim 1 , wherein a new image frame is processed to associate a face in the new image frame with a face that has previously been tracked if a facial image in the new image frame satisfies a predefined similarity metric with respect to the face that has previously been tracked.

9. The method of claim 1 , wherein a new image frame is processed to:
assign a name to a facial image in the new image frame if the facial image satisfies a predefined similarity metric with a given face in the second database.

10. The method of claim 1 , further comprising:
matching one or more images of a face of an unnamed person in the second database to images of known faces in the first database to obtain a name of the unnamed person.

11. The method of claim 1 , further comprising:
applying a maintenance process to remove from the second database the facial images corresponding to at least one individual of the subset of the individuals based at least in part on a time stamp the at least one individual last appeared in an image frame generated by the one or more cameras.

12. The method of claim 1 , further comprising:
applying an optimization process to one or more of add and remove the facial images corresponding to at least one individual of the subset of the individuals based on one or more time ranges specified for a given day.

13. A computer program product, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
maintaining a first database of facial images of a plurality of individuals;
maintaining a second database of facial images comprising a subset of the individuals from the first database of images, wherein the subset is obtained based on a probability of individuals appearing in one or more sequences of image frames at a given time, wherein the first data database and the second database are maintained on separate computing devices;
obtaining the one or more sequences of image frames generated by one or more cameras;
applying a face detection algorithm to at least one of the sequences of image frames to identify one or more faces in the one or more sequences of image frames;
applying facial recognition to at least one sequence of image frames using at least the second database to identify one or more individuals in the at least one sequence of image frames; and
in response to determining that the identified one or more faces comprises at least one new face that does not satisfy a predefined similarity metric with a given face in the second database: (i) adding a new identifier for the at least one new face to the second database and at least one new facial image of the at least one new face to the second database, (ii) tracking, based at least in part on the new identifier and the at least one new facial image, the at least one new face to obtain one or more additional facial images of the at least one new face, and (iii) in response to obtaining a specified number of facial images of the at least one new face, matching the facial images of the at least one new face to images of known faces in the first database.

14. The computer program product of claim 13 , wherein the second database is comprised of facial images of one or more of: (i) one or more individuals from a plurality of angles; (ii) one or more individuals that appeared in one or more prior image frames; and (iii) one or more individuals that appeared in at least one image frame generated by a plurality of cameras.

15. The computer program product of claim 13 , wherein a new image frame is processed to:
associate a face in the new image frame with a face that has previously been tracked if a facial image in the new image frame satisfies a predefined similarity metric with respect to the face that has previously been tracked.

16. The computer program product of claim 13 , further comprising the step of matching one or more images of a face of an unnamed person in the second database to images of known faces in the first database to obtain a name of the unnamed person.

17. An apparatus, comprising:
a memory; and
at least one processing device, coupled to the memory, the at least one processing device being configured:
to maintain a first database of facial images of a plurality of individuals;
to maintain a second database of facial images comprising a subset of the individuals from the first database of images, wherein the subset is obtained based on a probability of individuals appearing in one or more sequences of image frames at a given time;
to obtain the one or more sequences o

Assignees

• Emc Ip Holding Co Llc

Inventors

• Gefen Avitan
• Sagi Omer
• Savir Amihai

Classifications

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/25 Primary

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title