Method for determining validity of facial feature, and electronic device

Patent metadata

Field	Value
Publication number	US-12190633-B2
Application number	US-202017617893-A
Country	US
Kind code	B2
Filing date	Jun 9, 2020
Priority date	Jun 11, 2019
Publication date	Jan 7, 2025
Grant date	Jan 7, 2025

Abstract

A method for determining validity of a facial feature, and an electronic device are provided. The method includes: determining posture information and/or facial change information based on a facial feature of a first video image, where the posture information is used to represent deflection of a character's head indicated by the facial feature in the first video image, the facial change information is used to represent a position change status of the character's head in the first video image and a second video image, and the second video image is a frame of image preceding the first video image in a video stream to which the first video image belongs; and determining whether the facial feature is an invalid feature based on the posture information and/or the facial change information, and further determining whether to perform three-dimensional reconstruction processing on the character's face in the first video image.

First claim

What is claimed is:

1. A method for determining validity of a facial feature, comprising:
obtaining, by an electronic device, a facial feature of a first video image;
determining, by the electronic device, posture information and facial change information based on the facial feature of the first video image, wherein the posture information represents deflection of a character's head indicated by the facial feature in the first video image, the facial change information represents a position change status of the character's head in the first video image and a second video image, the facial change information comprises at least one of displacement of a position of the character's head in the first video image relative to a position of the character's head in the second video image, or an image similarity between a facial image of the first video image and a facial image of the second video image, the second video image is a frame of image preceding the first video image in a video stream to which the first video image belongs, and a quantity of video images between the second video image and the first video image is within a preset quantity range, wherein the facial feature comprises N groups of feature points, wherein N is a positive integer greater than or equal to 1, different groups of feature points belong to different facial organs, and wherein the displacement in the facial change information is determined based on:
determining, by the electronic device, a distance between a reference point of an i-th group of feature points in the first video image and a reference point of an i-th group of feature points in the second video image to obtain N groups of distances, wherein i represents a positive integer less than or equal to N; and
determining, by the electronic device, an average value of the N groups of distances as the displacement, or determining an average value of the normalized N groups of distances as the displacement; and
determining, by the electronic device, whether the facial feature is an invalid feature based on the posture information and the facial change information.

2. The method according to claim 1 , wherein the determining, by the electronic device, posture information and facial change information based on the facial feature of the first video image comprises:
determining, by the electronic device, the posture information and the facial change information based on the facial feature of the first video image when the facial feature does not include a feature point located in an edge area of the first video image.

3. The method according to claim 1 , wherein the posture information comprises at least one of horizontal deflection used to represent left or right turning of the character's head, vertical deflection used to represent raising or lowering of the character's head, and inclination used to represent a skew degree of the character's head.

4. The method according to claim 3 , wherein the determining, by the electronic device, whether the facial feature is an invalid feature based on the posture information and the facial change information comprises:
determining, by the electronic device, that the facial feature in the first video image is an invalid feature when it is determined that at least one of the following conditions is met:
the displacement is greater than a first threshold;
the image similarity is greater than a second threshold;
the horizontal deflection is greater than a third threshold;
the vertical deflection is greater than a fourth threshold; or
the inclination is greater than a fifth threshold.

5. The method according to claim 3 , wherein the facial feature comprises a left cheek feature, a right cheek feature, and a nose feature; and
wherein the horizontal deflection is determined based on:
determining, by the electronic device, a first distance between a reference point of the left cheek feature and a reference point of the nose feature, and a second distance between a reference point of the right cheek feature and a reference point of the nose feature; and
determining, by the electronic device, a ratio of the first distance to the second distance as the horizontal deflection, or determining a ratio of the second distance to the first distance as the horizontal deflection, wherein
the reference point of the left cheek feature is a preset feature point in the left cheek feature, the reference point of the nose feature is a preset feature point in the nose feature, and the reference point of the right cheek feature is a preset feature point in the right cheek feature; or
the reference point of the left cheek feature is a pixel corresponding to an average value of feature point coordinates comprised in the left cheek feature, the reference point of the nose feature is a pixel corresponding to an average value of feature point coordinates comprised in the nose feature, and the reference point of the right cheek feature is a pixel corresponding to an average value of feature point coordinates comprised in the right cheek feature.

6. The method according to claim 3 , wherein the facial feature comprises an eye feature, a chin feature, and a nose feature; and
wherein the vertical deflection is determined based on:
determining, by the electronic device, a third distance between a reference point of the eye feature and a reference point of the nose feature, and a fourth distance between a reference point of the chin feature and a reference point of the nose feature; and
determining, by the electronic device, a ratio of the third distance to the fourth distance as the horizontal deflection, or determining a ratio of the fourth distance to the third distance as the vertical deflection, wherein
the reference point of the eye feature is a preset feature point in the eye feature, the reference point of the nose feature is a preset feature point in the nose feature, and the reference point of the chin feature is a preset feature point in the chin feature; or
the reference point of the eye feature is a pixel corresponding to an average value of feature point coordinates comprised in the eye feature, the reference point of the nose feature is a pixel corresponding to an average value of feature point coordinates comprised in the nose feature, and the reference point of the chin feature is a pixel corresponding to an average value of feature point coordinates comprised in the chin feature.

7. The method according to claim 3 , wherein the facial feature comprises a left eye feature and a right eye feature; and
wherein the inclination is determined based on:
determining, by the electronic device, an angle in the horizontal direction of an offset of a connection line between a reference point of the left eye feature and a reference point of the right eye feature as the inclination, wherein
the reference point of the left eye feature is a pixel corresponding to an average value of feature point coordinates comprised in the left eye feature, and the reference point of the right eye feature is a pixel corresponding to an average value of feature point coordinates comprised in the right eye feature; or the reference point of the left eye feature is a left eye pupil feature point in the left eye feature, and the reference point of the right eye feature is a right eye pupil feature point in the right eye feature.

8. The method according to claim 1 , wherein the facial feature comprises M groups of feature points, wherein M is a positive integer greater than or equal to 1, and different groups of feature points belong to different facial organs; and
wherein the image similarity is determined based on:
determining, by the electronic device, a similarity between a j-th local feature image in the first video image

Assignees

• Honor Device Co Ltd

Inventors

• Chen Tianluo
• Qin Lei
• Zhou Heng
• Ding Xin
• Jiang Yongtao

Classifications

• G06V40/172

  Classification, e.g. identification · CPC title

• G06T2207/30201

  Face · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title