Method and device of registering face based on video data, and electronic whiteboard

Patent metadata

Field	Value
Publication number	US-11908235-B2
Application number	US-202017595605-A
Country	US
Kind code	B2
Filing date	Dec 25, 2020
Priority date	Dec 25, 2020
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

The embodiments of the present disclosure provide a method of registering a face based on video data, including: receiving video data; acquiring a first image frame sequence from the video data, wherein each image frame in the first image frame sequence includes a face detection frame containing a complete facial feature; determining whether each image frame reaches a preset definition or not according to a relative position of the face detection frame in the image frame; extracting a plurality of sets of facial features based on an image information of the plurality of face detection frames in response to determining that the image frame reaches the preset definition, and determining whether the faces represent an object or not according to the plurality of sets of facial features; and registering the object according to the first image frame sequence in response to determining that the faces represent the object.

First claim

What is claimed is:

1. A method of registering a face based on video data, comprising:
receiving video data;
acquiring a first image frame sequence from the video data, wherein each image frame in the first image frame sequence comprises a face detection frame containing a complete facial feature of a face;
determining, according to a relative position of the face detection frame in each image frame, whether the image frame reaches a preset definition or not;
extracting a plurality of sets of facial features based on an image information of a plurality of face detection frames in response to determining that the image frame reaches the preset definition, and determining whether the face represents an object or not according to the plurality of sets of facial features, wherein the object is a person; and
registering the object according to the first image frame sequence in response to determining that the face represents the object;
wherein the registering the object according to the first image frame sequence comprises: registering the object by using a designated image frame in the first image frame sequence as registration data;
wherein the acquiring a first image frame sequence from the video data comprises:
acquiring a plurality of image frames from the video data according to a sequence of capturing a video;
determining whether the plurality of image frames contains a face or not based on a face detection model; and
determining a face detection frame containing the face in each image frame of the plurality of image frames, in response to determining that the image frame contains the face; and

wherein the acquiring a first image frame sequence from the video data further comprises:
determining whether the image frame acquired contains a complete facial feature or not;
storing the image frame as a frame in the first image frame sequence in response to determining that the image frame contains the complete facial feature; and
ending the acquiring of image frames in response to determining that a predetermined number of image frames are stored in the first image frame sequence.

2. The method of claim 1 , wherein the determining whether the image frame acquired contains a complete facial feature or not comprises:
determining whether the face is a frontal face based on a face pose detection model;
determining whether the face is occluded or not based on a face occlusion detection model in response to determining that the face contained in the image frame is the frontal face;
determining that the image frame contains the complete facial feature, in response to determining that the face contained in the image frame is not occluded; and
determining that the image frame does not contain the complete facial feature, in response to determining that the face contained in the image frame is occluded.

3. The method of claim 1 , wherein the determining, according to a relative position of the face detection frame in each image frame, whether the image frame reaches a preset definition or not comprises:
determining a first ratio of an area of an intersection region of face detection frames in two image frames in the first image frame sequence to an area of a union region of the face detection frames in the two image frames; and
determining that the image frame reaches the preset definition, in response to the first ratio determined being greater than a first threshold.

4. The method of claim 3 , wherein the determining whether the faces represent an object or not according to the plurality of sets of facial features comprises:
determining a similarity between facial features in any two adjacent image frames in the first image frame sequence; and
determining that the faces represent the object, in response to the similarity determined being greater than a third threshold.

5. The method of claim 4 , wherein the facial feature comprises a facial feature vector, and wherein the determining a similarity between facial features in any two adjacent image frames in the first image frame sequence comprises:
determining a distance between the facial feature vectors in the two adjacent image frames in the first image frame sequence.

6. The method of claim 1 , wherein the determining, according to a relative position of the face detection frame in each image frame, whether the image frame reaches a preset definition or not comprises:
determining a first ratio of an area of an intersection region of face detection frames in two image frames in the first image frame sequence to an area of a union region of the face detection frames in the two image frames;
determining a second ratio of a number of the first ratio greater than the first threshold to a total number of the first ratio; and
determining that the image frame reaches the preset definition, in response to the second ratio being greater than or equal to a second threshold.

7. The method of claim 1 , further comprising:
storing the registration data obtained by registering the object according to the first image frame sequence as a face database; and
recognizing a face in the video data received, based on the face database.

8. The method of claim 7 , wherein the recognizing a face in the video data received, based on the face database comprises:
acquiring a second image frame sequence from the video data received, wherein each image frame in the second image frame sequence comprises a face detection frame containing a complete facial feature;
determining, according to a relative position of the face detection frame in each image frame, whether the image frame contains a living face or not;
extracting a facial feature based on the face detection frame in response to determining that the image frame contains the living face; and
determining whether the facial feature matches the registration data in the face database or not, so as to recognize the face.

9. The method of claim 1 , wherein the determining, according to a relative position of the face detection frame in each image frame, whether the image frame contains a living face or not comprises:
determining face detection frames meeting a coincidence condition of the plurality of face detection frames in each image frame;
determining a third ratio of a number of the face detection frames meeting the coincidence condition to a total number of the plurality of face detection frames;
determining that the face is a non-living face in response to the third ratio being greater than or equal to a fourth threshold; and
determining that the face is a living face in response to the third ratio being less than the fourth threshold.

10. The method of claim 9 , wherein the determining face detection frames meeting a coincidence condition of the plurality of face detection frames in each image frame comprises:
determining a fourth ratio of an area of an intersection region of any two face detection frames of the plurality of face detection frames to an area of each face detection frame of the two face detection frames;
determining that the two face detection frames are the face detection frames meeting the coincidence condition, in response to the fourth ratios determined being both greater than a fifth threshold; and
determining that the two face detection frames are not the face detection frames meeting the coincidence condition, in response to the fourth ratios determined being both less than the fifth threshold.

11. The method of claim 10 , wherein the determining, according to a relative position of the face detection frame in each image frame, whether the image frame contains a living face or not further comprises:
determining that the face is a non-living face in

Assignees

• Boe Technology Group Co Ltd

Inventors

• Xu Jingtao

Classifications

• G06V40/167 Primary

  using comparisons between temporally consecutive images · CPC title

• G06V10/22

  by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06V20/48

  Matching video sequences · CPC title