Systems and methods for fine tuning image classification neural networks

Patent metadata

Field	Value
Publication number	US-11763551-B2
Application number	US-202016807825-A
Country	US
Kind code	B2
Filing date	Mar 3, 2020
Priority date	Mar 3, 2020
Publication date	Sep 19, 2023
Grant date	Sep 19, 2023

Abstract

An authentication engine, residing at one or more computing machines, receives, from a vision device comprising one or more cameras, a probe image. The authentication engine generates, using a trained facial classification neural engine, one or more first labels for a person depicted in the probe image and a probability for at least one of the one or more first labels. The authentication engine determines that the probability is within a predefined low accuracy range. The authentication engine generates, using a supporting engine, a second label for the person depicted in the probe image. The supporting engine operates independently of the trained facial classification neural engine. The authentication engine further trains the facial classification neural engine based on the second label.

First claim

What is claimed is:

1. A system comprising:
processing circuitry; and
a memory storing instructions which, when executed by the processing circuitry, cause the processing circuitry to perform operations comprising:
receiving, from a vision device comprising one or more cameras, a probe image;
generating, using a trained facial classification neural engine, a first label for a person depicted in the probe image, the first label acting as an identifier of the person, and generating a probability for the first label, the probability corresponding to a confidence that the first label accurately identifies the person;
determining whether the probability is within a predefined high accuracy range;
if the probability is within the predefined high accuracy range, allowing access by the person to a physical location or an electronic resource; and
if the probability is not within the predefined high accuracy range:
allowing or denying access by the person to the physical location or electronic resource based on additional authentication information provided by the person; and
determining whether the probability is within a predefined low accuracy range, and if so:
generating, using a supporting engine, a second label for the person depicted in the probe image, the second label acting as an identifier of the person, wherein the supporting engine operates independently of the trained facial classification neural engine; and
further training the facial classification neural engine based on the second label.

2. The system of claim 1 , the operations further comprising:
using the further trained facial classification neural engine to identify one or more persons in visual data from the vision device; and
based on the identified one or more persons in the visual data, controlling access to the physical location or electronic resource for the one or more persons.

3. The system of claim 1 , wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises:
generating the second label based on an identity card or token provided by the person or based on a user identifier and password entered by the person.

4. The system of claim 1 , wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises:
generating the second label based on a combination of weak authentication factors, the weak authentication factors comprising one or more of: a height, a weight and a gait.

5. The system of claim 1 , wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises:
verifying, via at least one client computing device, a correct identification for the person depicted in the probe image.

6. The system of claim 5 , wherein verifying the correct identification comprises:
providing, for display at the at least one client computing device, the probe image and a plurality of possible identifications for the person; and
receiving, from the at least one client device, a selection of one of the possible identifications as the correct identification.

7. The system of claim 5 , wherein the at least one client computing device comprises an administrator client computing device and N employee client computing devices, wherein N is a positive integer greater than or equal to two, wherein verifying the correct identification comprises:
providing the probe image to at least a portion of the N employee client computing devices;
upon receiving, from at least M employee client computing devices, a consistent identification of the person: verifying that the consistent identification is the correct identification, wherein M is a positive integer between half of N and N; and
upon failing to receive, from the at least M employee client computing devices, the consistent identification of the person: providing the probe image to the administrator client computing device for verifying the correct identification via the administrator client computing device.

8. The system of claim 7 , wherein the N employee client computing devices are selected based on a corporate department or an office geographic location of at least one of a plurality of possible identifications.

9. The system of claim 1 , wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises:
providing the probe image to a training dataset for a semi-supervised learning facial classification engine;
training the semi-supervised learning facial classification engine using the training dataset;
generating, using the semi-supervised learning facial classification engine, the second label for the person depicted in the probe image and a probability value for the second label; and
adjusting the trained facial classification neural engine based on the trained semi-supervised learning facial classification engine.

10. The system of claim 9 , wherein providing the probe image to the training dataset for the semi-supervised learning facial classification engine is in response to determining that a quality of the probe image exceeds a quality threshold.

11. The system of claim 10 , wherein the quality of the probe image is computed using a quality measuring neural engine.

12. The system of claim 10 , wherein the quality of the probe image comprises a blurriness of the probe image.

13. The system of claim 9 , wherein generating, using the supporting engine, the second label for the person depicted in the probe image further comprises:
determining that the probability value for the second label is below a probability threshold; and
in response to the probability value for the second label being below the probability threshold: verifying, via at least one client computing device, a correct identification for the person depicted in the probe image.

14. The system of claim 1 , wherein the probe image is one of a plurality of images that track the person, the plurality of images being received from the vision device, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises:
determining, using the trained facial classification neural engine, that at least a threshold number of the plurality of images have a specified identification with a probability within a predefined high accuracy range; and
determining that the probe image has the specified identification based on the at least the threshold number of the plurality of images having the specified identification.

15. The system of claim 14 , the operations further comprising:
identifying the plurality of images that track the person based on timestamps associated with the plurality of images and a physical position of the person within a space depicted in the plurality of images.

16. A non-transitory machine-readable medium storing instructions which, when executed by processing circuitry of one or more computing machines, cause the processing circuitry to perform operations comprising:
receiving, from a vision device comprising one or more cameras, a probe image;
generating, using a trained facial classification neural engine, a first label for a person depicted in the probe image, the first label acting as an identifier of the person, and generating a probability for the first label, the probability corresponding to a confidence that the first label accurately identifies the person;
determining whether the probability is within a predefined high accuracy range;
if the probability is within the predefined high accuracy range, al

Assignees

• Assa Abloy Ab

Inventors

• Sachdeva Kapil
• Prevost Sylvain Jacques

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/7784 Primary

  based on feedback from supervisors · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title