Video processing method, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-12277689-B2
Application number	US-202218056155-A
Country	US
Kind code	B2
Filing date	Nov 16, 2022
Priority date	Jan 24, 2022
Publication date	Apr 15, 2025
Grant date	Apr 15, 2025

Abstract

Provided are a video processing method and apparatus, an electronic device, a storage medium and a program product. The method includes steps described below. A to-be-processed video is acquired, and a first face key point of a first target object in the to-be-processed video is recognized, where the first face key point corresponds to a mask key point in a three-dimensional human face mask effect; and the three-dimensional human face mask effect is added to video frames of the to-be-processed video to obtain a target video, where the mask key point in the three-dimensional human face mask effect moves with the corresponding first face key point.

First claim

What is claimed is:

1. A video processing method, comprising:
acquiring a first video, and recognizing at least one first face key point of a first target object in the first video, wherein the at least one first face key point corresponds to at least one mask key point in a three-dimensional human face mask effect; and
adding the three-dimensional human face mask effect to video frames of the first video to obtain a second video, wherein the at least one mask key point in the three-dimensional human face mask effect moves with a corresponding first face key point;
wherein adding the three-dimensional human face mask effect to the video frames of the first video comprises:
determining first position data of the at least one first face key point in a current first video frame in the first video, and acquiring a first three-dimensional human face model corresponding to the current first video frame;
updating a position of each patch vertex in the first three-dimensional human face model according to the first position data to obtain a second three-dimensional human face model corresponding to the current first video frame;
performing mapping rendering on the second three-dimensional human face model to obtain a current three-dimensional human face mask effect; and
adding the current three-dimensional human face mask effect to the current first video frame to obtain a current second video frame; and

acquiring to-be-rendered texture information of the second three-dimensional human face model, comprising:
determining a to-be-rendered material corresponding to the current first video frame, and acquiring texture information of the to-be-rendered material as the to-be-rendered texture information of the second three-dimensional human face model.

2. The method according to claim 1 , wherein adding the current three-dimensional human face mask effect to the current first video frame to obtain the current second video frame comprises:
determining a preset effect display position corresponding to the current first video frame; and
adding the current three-dimensional human face mask effect to the preset effect display position of the current first video frame to obtain the current second video frame.

3. The method according to claim 2 , before acquiring the first video, further comprising:
acquiring an original three-dimensional human face model and unrenderable part information, wherein the unrenderable part information is information of an unrenderable part in the original three-dimensional human face model;
deleting a patch corresponding to the unrenderable part in the original three-dimensional human face model according to the unrenderable part information to obtain a target three-dimensional human face model; and
using the target three-dimensional human face model as a first three-dimensional human face model corresponding to a first video frame in the first video.

4. The method according to claim 1 , wherein adding the current three-dimensional human face mask effect to the current first video frame to obtain the current second video frame comprises:
recognizing a second face key point of a second target object in the current first video frame;
determining a current effect display position according to a position of the second face key point in the current first video frame; and
adding the current three-dimensional human face mask effect to the current effect display position of the current first video frame to obtain the current second video frame.

5. The method according to claim 4 , before acquiring the first video, further comprising:
acquiring an original three-dimensional human face model and unrenderable part information, wherein the unrenderable part information is information of an unrenderable part in the original three-dimensional human face model;
deleting a patch corresponding to the unrenderable part in the original three-dimensional human face model according to the unrenderable part information to obtain a target three-dimensional human face model; and
using the target three-dimensional human face model as a first three-dimensional human face model corresponding to a first video frame in the first video.

6. The method according to claim 1 , wherein performing the mapping rendering on the second three-dimensional human face model comprises:
performing the mapping rendering on the second three-dimensional human face model according to the to-be-rendered texture information.

7. The method according to claim 6 , wherein performing the mapping rendering on the second three-dimensional human face model according to the to-be-rendered texture information comprises:
generating the face texture mapping according to the to-be-rendered texture information; and
performing rendering on the second three-dimensional human face model based on the face texture mapping.

8. The method according to claim 6 , before acquiring the first video, further comprising:
acquiring an original three-dimensional human face model and unrenderable part information, wherein the unrenderable part information is information of an unrenderable part in the original three-dimensional human face model;
deleting a patch corresponding to the unrenderable part in the original three-dimensional human face model according to the unrenderable part information to obtain a target three-dimensional human face model; and
using the target three-dimensional human face model as a first three-dimensional human face model corresponding to a first video frame in the first video.

9. The method according to claim 1 , wherein acquiring the to-be-rendered texture information of the second three-dimensional human face model comprises:
recognizing a third target object in the current first video frame, and acquiring face texture information of the third target object as the to-be-rendered texture information of the second three-dimensional human face model.

10. The method according to claim 9 , wherein acquiring the face texture information of the third target object comprises:
separately determining coordinate information corresponding to each patch vertex in the second three-dimensional human face model in face texture mapping;
performing face sampling on the third target object according to the coordinate information to obtain face sampling data of the third target object; and
generating the face texture information of the third target object according to the face sampling data.

11. The method according to claim 10 , wherein performing the mapping rendering on the second three-dimensional human face model according to the to-be-rendered texture information comprises:
generating the face texture mapping according to the to-be-rendered texture information; and
performing rendering on the second three-dimensional human face model based on the face texture mapping.

12. The method according to claim 9 , wherein performing the mapping rendering on the second three-dimensional human face model according to the to-be-rendered texture information comprises:
generating the face texture mapping according to the to-be-rendered texture information; and
performing rendering on the second three-dimensional human face model based on the face texture mapping.

13. The method according to claim 9 , before acquiring the first video, further comprising:
acquiring an original three-dimensional human face model and unrenderable part information, wherein the unrenderable part information is information of an unrenderable part in the original three-dimensional human face model;
deleting a patch corresponding to the unrenderable part in the original three-dimensional human fa

Assignees

• Lemon Inc

Inventors

• Zhang Jingcong
• Schager Nathanael
• Tang Xiaocheng
• Gualtieri James
• Lv Yang
• Huang Zhe
• Cai Zeyong
• Wang Jing
• Liu Xiaoyu
• Luo Nite
• Meng Julia
• Cheng Haiying
• Tan Qinzi

Classifications

• G06T7/40

  Analysis of texture (depth or shape recovery from texture G06T7/529) · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/30201

  Face · CPC title

• G11B27/036

  Insert-editing · CPC title

• H04N21/816

  involving special video data, e.g 3D video · CPC title