Key point positioning method, terminal, and computer storage medium

Patent metadata

Field	Value
Publication number	US-10990803-B2
Application number	US-201816222941-A
Country	US
Kind code	B2
Filing date	Dec 17, 2018
Priority date	Aug 10, 2016
Publication date	Apr 27, 2021
Grant date	Apr 27, 2021

Abstract

When a target image is captured, the device provides a portion of the target image within a target detection region to a preset first model set to calculate positions of face key points and a first confidence value. The face key points and the first confidence value are output by the first model set for a single input of the portion of the first target image into the first model set. When the first confidence value meets a first threshold corresponding to whether the target image is a face image, the device obtains a second target image corresponding to the positions of the first face key points; the device inputs the second target image into the first model set to calculate a second confidence value, the second confidence value corresponds to accuracy key point positioning, and outputs the first key points if the second confidence value meets a second threshold.

First claim

What is claimed is:

1. A facial key point positioning method, comprising:
at a device having one or more processors, a camera and memory:
capturing, using the camera, a first target image;
generating a target detection region for facial recognition on the first target image according to a preset configuration;
displaying the target detection region for facial recognition at a location in a field of view of the camera represented in a user interface of an image capturing application associated with the camera;
in accordance with a determination that the first target image is captured, providing a single input of a portion of the first target image within the target detection region to an input layer of a preset first model set;
calculating, using the preset first model set, positions of a first set of face key points and a first confidence value, wherein the first set of face key points and the first confidence value are output by the preset first model set for the single input of the portion of the first target image into the preset first model set;
in accordance with a determination that the first confidence value meets a first preset threshold, the first preset threshold corresponds to whether the first target image is a face image:
obtaining, using the camera, a second target image corresponding to the positions of the first set of face key points in the target detection region to determine a difference between the first target image and the second target image;
inputting the second target image into the preset first model set to calculate a second confidence value, wherein the second confidence value corresponds to an accuracy of the positions of the first set of face key points in the first target image, and the second confidence value is used for indicating whether and how the device is to be adjusted to minimize the difference between the first target image and the second target image; and
in accordance with a determination that the second confidence value meets a second preset threshold, outputting the positions of the first set of face key points as final face key points of the first target image in the user interface, wherein a building process for the preset first model set includes:
obtaining first training data for face key point positioning, the first training data comprising pair-wise correspondences between face images and respective sets of pre-calibrated key point coordinates; and
obtaining second training data for confidence, the second training data being a classification set of previously acquired target images, the classification set includes labels of facial recognition for face images and non-face or abnormal face images.

2. The method according to claim 1 , wherein the preset first model set comprises a shared branch, a key point positioning branch, and a confidence branch, wherein the key point positioning branch and the confidence branch are respectively cascaded from the shared branch, an input of the shared branch is an input branch of the preset first model set; the key point positioning branch and the confidence branch are two output branches of the preset first model set, and the shared branch is an image feature data model in the building process of the preset first model set.

3. The method according to claim 2 , wherein the building process of the preset first model set includes:
training model parameters of the shared branch and the key point positioning branch according to the first training data to obtain a model of the shared branch and a model of the key point positioning branch at the same time;
training model parameters of the confidence branch according to the second training data to obtain a model of the confidence branch; and
connecting the trained model of the key point positioning branch and the trained model of the confidence branch respectively to the model of the shared branch, to generate the preset first model set.

4. The method according to claim 3 , wherein providing a portion of the first target image within the target detection region to an input layer of a preset first model set to calculate positions of a first set of face key points and a first confidence value comprises:
inputting the portion of the first target image into the model of the shared branch, to extract first image features;
inputting the first image features into the model of the key point positioning branch, to calculate the positions of the first face key points; and
inputting the first image features into the model of the confidence branch, to calculate the first confidence value.

5. The method according to claim 3 , wherein inputting the second target image into the preset first model set to calculate a second confidence value comprises:
inputting the second target image into the model of the shared branch, to extract second image features; and
inputting the second image features into the model of the confidence branch, to calculate the second confidence value.

6. The method according to claim 3 , including:
before training the model of the key point positioning branch and the model of the confidence branch:
performing error correction on the model of the key point positioning branch and the model of the confidence branch respectively according to a preset policy; and
determining a corrected model of the key point positioning branch and a corrected model of the confidence branch, for face key point positioning and confidence determining.

7. The method according to claim 1 , including:
after calculating the positions of the first face key points and the first confidence value:
determining, in accordance with a determination that the first confidence value does not meet the first preset threshold, that the positioning of the positions of the first face key points fails; and ending the positioning of the face key points based on the first target image.

8. A device, comprising:
one or more processors;
memory;
a camera
a display; and
a plurality of instructions stored in the memory that, when executed by the one or more processors, cause the one or more processors to perform the following operations:
capturing, using the camera, a first target image;
generating a target detection region for facial recognition on the first target image according to a preset configuration;
displaying the target detection region for facial recognition at a location in a field of view of the camera represented in a user interface of an image capturing application associated with the camera;
in accordance with a determination that the first target image is captured, providing a single input of a portion of the first target image within the target detection region to an input layer of a preset first model set;
calculating, using the preset first model set, positions of a first set of face key points and a first confidence value, wherein the first set of face key points and the first confidence value are output by the preset first model set for the single input of the portion of the first target image into the preset first model set;
in accordance with a determination that the first confidence value meets a first preset threshold, the first preset threshold corresponds to whether the first target image is a face image:
obtaining, using the camera, a second target image corresponding to the positions of the first set of face key points in the target detection region to determine a difference between the first target image and the second target image;
inputting the second target image into the preset first model set to calculate a second confidence value, wherein the second confidence value corresponds to an accuracy of the positions of the first set of face key points in the first target image, and the second confid

Assignees

• Tencent Tech Shenzhen Co Ltd
• Tencent Tech Shenzhen
• Company Ltd

Inventors

• Wang Chengjie
• Li Jilin
• Zhao Yandan
• Ni Hui
• Wang Yabiao
• Zhao Ling

Classifications

• G06V10/774 Primary

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V40/165 Primary

  using facial parts and geometric relationships · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title