Face image processing method and apparatus, image device, and storage medium

Patent metadata

Field	Value
Publication number	US-11941854-B2
Application number	US-202117203171-A
Country	US
Kind code	B2
Filing date	Mar 16, 2021
Priority date	Aug 28, 2019
Publication date	Mar 26, 2024
Grant date	Mar 26, 2024

Abstract

Provided are a face image processing method and apparatus, an image device, and a storage medium. The face image processing method includes: acquiring first-key-point information of a first face image; performing position transformation on the first-key-point information to obtain second-key-point information conforming to a second facial geometric attribute, the second facial geometric attribute being different from a first facial geometric attribute corresponding to the first-key-point information; and performing facial texture coding processing by utilizing a neural network and the second-key-point information to obtain a second face image.

First claim

The invention claimed is:

1. A method for face image processing, comprising:
acquiring first-key-point information of a first face image;
performing position transformation on the first-key-point information to obtain second-key-point information conforming to a second facial geometric attribute, the second facial geometric attribute being different from a first facial geometric attribute corresponding to the first-key-point information; and
performing facial texture coding processing by utilizing a neural network and the second-key-point information to obtain a second face image,
wherein performing facial texture coding processing by utilizing the neural network and the second-key-point information to obtain the second face image comprises:
generating a mask map of a target face based on the second-key-point information, values of pixels in a face image area in the mask map being a first predetermined value, values of pixels outside the face image area being a second predetermined value, and the face image area being defined by each second key point describing a facial contour;
fusing the mask map and the first face image to generate a fused face image, the face image area in the first face image being reserved in the fused face image;
generating a contour map of geometric attributes of the target face based on the second-key-point information, values of pixels on a contour line of each part in the contour map being a third predetermined value, values of pixels other than the pixels on the contour line of each part being a fourth predetermined value, and the contour line of each part being defined by second key points describing each face part; and
inputting the fused face image and the contour map into the neural network for facial texture coding to obtain the second face image.

2. The method according to claim 1 , wherein
after performing position transformation on the first-key-point information to obtain the second-key-point information conforming to the second facial geometric attribute, the method further comprises: adjusting the first face image based on the second-key-point information to obtain a third face image; and
after performing facial texture coding processing by utilizing the neural network and the second-key-point information to obtain the second face image, the method further comprises:
fusing the second face image and the third face image to obtain a fourth face image.

3. The method according to claim 2 , further comprising:
replacing the first face image with the fourth face image or the second face image.

4. The method according to claim 1 , wherein the first face image is contained in a predetermined image, the method further comprising:
fusing a fourth face image with a background area other than the first face image in the predetermined image to generate an updated image.

5. The method according to claim 1 , further comprising:
determining the first facial geometric attribute based on the first-key-point information; and
acquiring the second facial geometric attribute;
wherein performing position transformation on the first-key-point information to obtain the second-key-point information conforming to the second facial geometric attribute comprises:
performing position transformation on the first-key-point information based on geometric attribute transformation parameters corresponding to both the first facial geometric attribute and the second facial geometric attribute to obtain the second-key-point information.

6. The method according to claim 5 , wherein at least one of different first facial geometric attributes correspond to different geometric attribute transformation parameters; or different second facial geometric attributes correspond to different geometric attribute transformation parameters.

7. The method according to claim 1 , wherein performing facial texture coding processing by utilizing the neural network and the second-key-point information to obtain the second face image further comprises:
inputting the contour map into the neural network for facial texture coding to obtain the second face image.

8. An apparatus for face image processing, comprising:
a processor; and
a memory configured to store instructions executable by the processor;
wherein the processor is configured to:
acquire first-key-point information of a first face image;
perform position transformation on the first-key-point information to obtain second-key-point information conforming to a second facial geometric attribute, the second facial geometric attribute being different from a first facial geometric attribute corresponding to the first-key-point information; and
perform facial texture coding processing by utilizing a neural network and the second-key-point information to obtain a second face image,

wherein the processor is further configured to:
generate a mask map of a target face based on the second-key-point information, values of pixels in a face image area in the mask map being a first predetermined value, values of pixels outside the face image area being a second predetermined value, and the face image area being defined by each second key point describing a facial contour;
fuse the mask map and the first face image to generate a fused face image, the face image area in the first face image being reserved in the fused face image;
generate a contour map of geometric attributes of the target face based on the second-key-point information, values of pixels on a contour line of each part in the contour map being a third predetermined value, values of pixels other than the pixels on the contour line of each part being a fourth predetermined value, and the contour line of each part being defined by second key points describing each face part; and
input the fused face image and the contour map into the neural network for facial texture coding to obtain the second face image.

9. The apparatus according to claim 8 , wherein the processor is further configured to:
after performing position transformation on the first-key-point information to obtain the second-key-point information conforming to the second facial geometric attribute, adjust the first face image based on the second-key-point information to obtain a third face image; and
after performing facial texture coding processing by utilizing the neural network and the second-key-point information to obtain the second face image, fuse the second face image and the third face image to obtain a fourth face image.

10. The apparatus according to claim 9 , wherein the processor is further configured to:
replace the first face image with the fourth face image or the second face image.

11. The apparatus according to claim 8 , wherein the first face image is contained in a predetermined image, and the processor is further configured to:
fuse a fourth face image with a background area other than the first face image in the predetermined image to generate an updated image.

12. The apparatus according to claim 8 , wherein the processor is further configured to:
determine the first facial geometric attribute based on the first-key-point information;
acquire the second facial geometric attribute of the target face, and
perform position transformation on the first-key-point information based on geometric attribute transformation parameters corresponding to both the first facial geometric attribute and the second facial geometric attribute to obtain the second-key-point information.

13. The apparatus according to claim 12 , wherein at least one of different first facial geometric attributes correspond to different geometric attribute transformation par

Assignees

• Beijing Sensetime Tech Development Co Ltd

Inventors

• Wu Wenyan
• Qian Chen
• Sun Keqiang
• Wu Qianyi
• Xu Yuanyuan

Classifications

• G06T9/002 Primary

  using neural networks · CPC title

• G06N3/02

  Neural networks · CPC title

• G06T3/00

  Geometric image transformations in the plane of the image · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/40

  Analysis of texture (depth or shape recovery from texture G06T7/529) · CPC title