Identity recognition utilizing face-associated body characteristics

Patent metadata

Field	Value
Publication number	US-12154386-B2
Application number	US-202217743416-A
Country	US
Kind code	B2
Filing date	May 12, 2022
Priority date	May 14, 2021
Publication date	Nov 26, 2024
Grant date	Nov 26, 2024

Abstract

Techniques are disclosed for providing a notification indicating an identity of a first person based on face-associated body characteristics. For example, a device performs facial recognition to identify the identity of the first person shown in a first video feed. The device also identifies and stores physical characteristic information of the first person from the first video feed, the stored information associated with the identity of the first person based on the recognized face. Subsequently, the device receives a second video feed showing a second person whose face is determined to not be recognized by the device. The device compares the stored physical characteristic information of the first person with additional physical characteristic information of the second person shown in the second video feed. Based on the comparison, the device provides a notification indicating whether the identity of the second person corresponds to the identity of the first person.

First claim

What is claimed is:

1. A method, comprising:
receiving, by at least one device of a plurality of devices and from a camera, a first plurality of video frames during a first time period, at least one of the frames of the first plurality of video frames including a face of a first person;
identifying, by the at least one device of the plurality of devices, an identity of the first person based at least in part on recognizing the face in the at least one frame of the first plurality of video frames;
identifying, by the at least one device of the plurality of devices, one or more physical characteristics of the first person from at least a frame of the first plurality of video frames that included the face;
storing, by the at least one device of the plurality of devices, a physical characteristic information corresponding to the identified one or more physical characteristics, the physical characteristic information stored in association with the identity of the first person based at least in part on the recognized face shown in the at least one frame of the first plurality of video frames;
receiving, by the at least one device of the plurality of devices, a second plurality of video frames during a second time period, the second plurality of video frames including a second person where a face of the second person is determined to not be recognized by the at least one device;
comparing, by the at least one device of the plurality of devices, additional physical characteristic information of the second person identified in the second plurality of video frames with the stored physical characteristic information associated with the identity of the first person; and
providing, by the at least one device of the plurality of devices, a notification indicating whether an identity of the second person corresponds to the identity of the first person based at least in part on the comparison.

2. The method of claim 1 , wherein the one or more physical characteristics of the first person are associated with at least one of a torso, a gait, one or more arms, one or more legs, or a body shape of the first person.

3. The method of claim 1 , wherein the one or more physical characteristics of the first person are associated with at least one of a texture or color of at least one of a body of the first person or an article of clothing worn by the first person.

4. The method of claim 1 , wherein the additional physical characteristic information corresponds to one or more physical characteristics of the second person that are a same type of the one or more physical characteristics of the first person.

5. The method of claim 1 , wherein the physical characteristic information corresponds to a bodyprint of the first person, the bodyprint including a multidimensional vector, a dimension of the vector associated with one or more of the physical characteristics of the first person.

6. The method of claim 1 , wherein the comparison is performed using a machine learning model that is trained to associate a first bodyprint of the first person with a second bodyprint of the first person.

7. The method of claim 1 , wherein providing the notification further comprises:
determining, by the at least one device of the plurality of devices, a confidence score based at least in part on the comparison;
determining, by the at least one device of the plurality of devices, that the confidence score matches a threshold metric; and
providing, by the at least one device of the plurality of devices, the notification that indicates that the identity of the second person matches the identity of the first person.

8. The method of claim 1 , further comprising:
storing, by the at least one device of the plurality of devices, information that corresponds to the face of the first person in association with the identified identity.

9. The method of claim 1 , further comprising:
storing, by the at least one device of the plurality of devices, a cluster of bodyprints of the first person based at least in part on the first plurality of video frames, a bodyprint of the cluster being associated with a particular frame of the first plurality of video frames and selected for inclusion among the cluster of bodyprints based at least in part on an information gain associated with the bodyprint, the information gain used to perform the comparison between the additional physical characteristic information of the second person and the physical characteristic information of the first person.

10. The method of claim 1 , wherein the plurality of devices includes a first device and a second device that are associated with a home environment setting, the method further comprising:
generating, by the first device, a bodyprint of the first person from a frame of the first plurality of video frames;
determining, by the first device, that the bodyprint will be added to a cluster of bodyprints associated with the first person based at least in part on an information gain associated with the bodyprint; and
transmitting, by the first device, the bodyprint to the second device, the bodyprint being stored by the second device among a second cluster of bodyprints maintained by the second device, wherein respective clusters of bodyprints are synchronized between the first device and the second device.

11. The method of claim 1 , wherein the at least one device receives the second plurality of video frames from a second camera, and wherein the camera is positioned in association with an entrance to a home environment and the second camera is positioned in association with an interior corridor of the home environment.

12. The method of claim 1 , further comprising:
determining, by the at least one device of the plurality of devices, to update a cluster of bodyprints associated with the first person, a bodyprint of the cluster associated with the stored physical characteristic information associated with the first person.

13. The method of claim 12 , wherein the cluster of bodyprints is updated based at least in part on determining a start of a new time period.

14. The method of claim 13 , further comprising:
receiving, by the at least one device of the plurality of devices, a third plurality of video frames during a third time period;
determining, by the at least one device of the plurality of devices, that at least one of the one or more identified physical characteristics of the first person have changed based at least in part on analyzing the third plurality of video frames; and
determining, by the at least one device of the plurality of devices, to update the cluster of bodyprints associated with the first person based at least in part on determining that at least one of the one or more identified physical characteristics of the first person have changed.

15. The method of claim 12 , wherein the physical characteristic information corresponds to a bodyprint of the first person, wherein a cluster of bodyprints of the first person are generated based at least in part on the first plurality of video frames, and wherein the cluster of bodyprints, respectively, are associated with the identity of the first person based at least in part on the recognition of the face in the at least one frame of the first plurality of video frames.

16. The method of claim 1 , wherein the camera is a separate device that is communicatively connected to the at least one device of the plurality of devices, and wherein the camera is configured to transmit the first plurality of video frames to the at least one device.

17. A device, comprising:
a memory comprising com

Assignees

• Apple Inc

Inventors

• Gupta Nitin
• Zhu Jingwen
• Jin Jonghoon
• Edwards Andrew C
• Chabert Floris
• Sharma Vinay
• Dahlkamp Hendrik

Classifications

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V40/103 Primary

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V40/50

  Maintenance of biometric data or enrolment thereof · CPC title

• G06V40/70 Primary

  Multimodal biometrics, e.g. combining information from different biometric modalities · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title