Identifying objects within images from different sources

Patent metadata

Field	Value
Publication number	US-11514717-B2
Application number	US-202017033423-A
Country	US
Kind code	B2
Filing date	Sep 25, 2020
Priority date	Jun 3, 2020
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

Techniques are disclosed for providing a notification that a person is at a particular location. For example, a resident device may receive from a user device an image that shows a face of a first person, the image being captured by a first camera of the user device. The resident device may also receive, from another device having a second camera, a second image showing a portion of a face of a second person, the second camera having a viewable area showing a particular location. The resident device may determine a score indicating a level of similarity between a first set of characteristics associated with the face of the first person and a second set of characteristics associated with the face of a second person. The resident device may then provide to the user device a notification based on determining the score.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, by a resident device from a remote server, an image cropping generated from an image of a plurality of images stored on a local repository of a user device, the image cropping comprising a portion of a face of a first person, the plurality of images captured by a first camera of the user device, the user device being configured to encrypt the image cropping and transmit the image cropping to the remote server;
receiving, by the resident device and from another device comprising a second camera, a second image comprising a portion of a face of a second person, the second camera having a viewable area comprising a particular location associated with the resident device;
determining, by the resident device, a score that corresponds to a level of similarity between a first set of characteristics associated with the face of the first person and a second set of characteristics associated with the face of the second person;
determining, by the resident device, whether the first person is the second person based at least in part on the score;
providing, by the resident device and to the user device, a notification based at least in part on the determination; and
receiving, by the resident device, an encryption key operable for decrypting the image cropping, the encryption key being shared with the resident device and not shared with the remote server.

2. The computer-implemented method of claim 1 , wherein the plurality of images is managed as part of a library of images comprising contacts associated with the user device.

3. The computer-implemented method of claim 1 , wherein the image is associated with a first level of image quality and the second image is associated with a second level of image quality that is different from the first level of image quality.

4. The computer-implemented method of claim 1 , wherein determining the score further comprises:
generating, by a trained model of the resident device, a first faceprint of the face of the first person based at least in part on the image cropping, the first faceprint corresponding to a multidimensional vector, a dimension of the vector associated with a characteristic of the first set of characteristics of the face of the first person.

5. The computer-implemented method of claim 1 , further comprising:
determining, by the resident device, that the first person is the second person based at least in part on the score; and
determining, by the resident device, not to provide the notification to the user device based at least in part on determining that the first person is the second person.

6. The computer-implemented method of claim 1 , further comprising:
determining, by the resident device, that the first person is not the second person based at least in part on the score; and
providing, by the resident device and to the user device, the notification that indicates that the second person is not a contact associated with the user device.

7. The computer-implemented method of claim 1 , further comprising:
maintaining, by the resident device, a face quality metric that indicates a level of quality associated with the second set of characteristics associated with the face of the second person, the face quality metric operable for determining whether a particular face of a person is recognizable or unrecognizable; and
determining, by the resident device, that the face of the second person is recognizable based at least in part determining that the level of quality indicated by the face quality metric matches a threshold; and
determining, by the resident device, whether the first person is the second person based at least in part on determining that the face of the second person is recognizable.

8. A first device, comprising:
a memory configured to store computer-executable instructions; and
one or more processors in communication with the memory and configured to access the memory and execute the computer-executable instructions to, at least:
receive from a remote server an image cropping generated from an image of a plurality of images stored on a local repository of a user device, the image cropping comprising a portion of a face of a first person, the plurality of images captured by a first camera of the user device, the user device being configured to encrypt the image cropping and transmit the image cropping to the remote server;
receive from another device comprising a second camera, a second image comprising a portion of a face of a second person, the second camera having a viewable area comprising a particular location associated with the first device;
determine a score that corresponds to a level of similarity between a first set of characteristics associated with the face of the first person and a second set of characteristics associated with the face of the second person;
determine whether the first person is the second person based at least in part on the score;
provide to the user device a notification based at least in part on the determination; and
receive, by the first device, an encryption key operable for decrypting the image cropping, the encryption key being shared with the first device and not shared with the remote server.

9. The first device of claim 8 , wherein the plurality of images, respectively, comprises a particular portion of the face of the first person, the image being one of a subset of the plurality of images, the image included in the subset based at least in part on an information gain associated with the portion of the face of the first person, the information gain used to perform a facial recognition of the face of the first person.

10. The first device of claim 9 , wherein the image cropping is one of a plurality of image croppings that are received by the first device from the remote server, the plurality of image croppings respectively generated from the subset of the plurality of images and operable as a reference set of image croppings.

11. The first device of claim 10 , wherein the instructions to determine the score comprise additional instructions to:
generate, by a trained model of the first device, a first faceprint of the face of the first person based at least in part on the plurality of image croppings; and
compare, by the trained model of the first device, the first faceprint with the second set of characteristics associated with the face of the second person.

12. The first device of claim 11 , wherein the instructions to compare the first faceprint with the second set of characteristics associated with the face of the second person comprise additional instructions to:
generate, by the trained model, a second faceprint of the face of the second person based at least in part on the second image, the second faceprint associated with the second set of characteristics; and
determine, by the trained model, a level of similarity between the first faceprint and the second faceprint.

13. The first device of claim 8 , wherein the one or more processors further execute the instructions to:
receive from the remote server a new image cropping showing another portion of the face of the first person, the new image cropping determined to provide an additional information gain;
include the new image cropping in an updated plurality of image croppings based at least in part on the additional information gain provided by the new image cropping; and
generate an updated faceprint of the face of the first person based at least in part on the updated plurality of image croppings.

14. The first device of claim 8 , wherein the instructions to determin

Assignees

• Apple Inc

Inventors

• Dahlkamp Hendrik
• Sharma Vinay
• Gupta Nitin
• Chabert Floris
• Edwards Andrew C
• Williams Mitchell
• Jin Jonghoon
• Fraioli Nicholas M
• Krishnan Aravindhan K
• Coffman Patrick L

Classifications

• G06V40/173 Primary

  face re-identification, e.g. recognising unknown faces across different face tracks · CPC title

• G06V20/53

  Recognition of crowd images, e.g. recognition of crowd congestion · CPC title

• G06V10/803

  of input or preprocessed data · CPC title

• G06V40/172 Primary

  Classification, e.g. identification · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title