Video processing method, apparatus and storage medium

Patent metadata

Field	Value
Publication number	US-11310443-B2
Application number	US-202117244703-A
Country	US
Kind code	B2
Filing date	Apr 29, 2021
Priority date	Jul 28, 2020
Publication date	Apr 19, 2022
Grant date	Apr 19, 2022

Abstract

The present disclosure relates to a video processing method and apparatus, and a storage medium. The method is applied to a terminal and includes: a background frame for a time static special effect is determined from video frames in a video to be processed; for each of the video frames in the video, an image area, where a target object is located is acquired from the respective video frame and the image area is fused into the background frame to generate a special effect frame with the time static special effect.

First claim

The invention claimed is:

1. A video processing method, implemented by a terminal, the method comprising:
selecting, from video frames in a video to be processed, a background frame for a static special effect; and
for each of the video frames in the video, acquiring an image area, where a target object is located, from the respective video frame, and fusing the image area into the background frame, to generate a special effect frame with the static special effect,
wherein selecting, from the video frames in the video to be processed, the background frame for the static special effect comprises:
determining a first frame after the video to be processed enters a time static mode in which the static special effect is formed, as the background frame for the static special effect.

2. The method of claim 1 , wherein fusing the image area into the background frame, to generate the special effect frame with the static special effect comprises:
for each of video frames starting from a start frame in the video, fusing the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect; or,
for each of video frames between a start frame and an end frame in the video, fusing the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect.

3. The method of claim 2 , further comprising:
displaying the video frames in the video when being recorded;
in response to a number of video frames in a predetermined storage space not exceeding a storage amount threshold, buffering the video frames starting from the start frame; and
fusing the image area, which is acquired from each of the buffered video frames, into the background frame to generate and display the special effect frame.

4. The method of claim 2 , further comprising:
detecting a posture of the target object in each of the video frames;
determining a video frame, in which the posture of the target object is detected as a first predetermined posture, to be the start frame; and
determining a video frame, in which the posture of the target object is detected as a second predetermined posture, to be the end frame.

5. The method of claim 1 , wherein fusing the image area into the background frame to generate the special effect frame with the static special effect comprises:
overlaying an area at a target position in the background frame with the image area; and
feathering edges of the image area with which the background frame was overlaid to form the special effect frame.

6. The method of claim 5 , wherein overlaying the area at the target position in the background frame with the image area comprises:
determining an alignment parameter according to the position of the target object in each of the video frames; and
overlaying, according to the alignment parameter, the area at the target position in the background frame with the image area of the target object.

7. The method of claim 6 , wherein determining the alignment parameter according to the position of the target object in each of the video frames comprises:
selecting a video frame from the video frames in the video as a reference frame; and
determining the alignment parameter according to a position offset of the target object in the video frame relative to the target object in the reference frame.

8. A terminal, comprising:
a processor; and
a memory for storing executable instructions executable by the processor,
wherein the processor is arranged to:
select, from video frames in a video to be processed, a background frame for a static special effect; and
for each of the video frames in the video, acquire, through a multimedia component, an image area, where a target object is located, from the respective video frame and fuse the image area into the background frame, to generate a special effect frame with the static special effect,
wherein the processor is further arranged to:
determine a first frame after the video to be processed enters a time static mode in which the static special effect is formed, as the background frame for the static special effect.

9. The terminal of claim 8 , wherein the processor is further arranged to:
for each of video frames starting from a start frame in the video, fuse the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect; or,
for each of video frames between a start frame and an end frame in the video, fuse the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect.

10. The terminal of claim 9 , wherein the processor is further arranged to:
display, through the multimedia component, the video frames in the video when being recorded;
in response to a number of video frames in a predetermined storage space not exceeding a storage amount threshold, buffer the video frames starting from the start frame; and
fuse the image area, which is acquired from each of the buffered video frames, into the background frame to generate and display the special effect frame.

11. The terminal of claim 9 , wherein the processor is further arranged to:
detect a posture of the target object in each of the video frames;
determine a video frame, in which the posture of the target object is detected as a first predetermined posture, to be the start frame; and
determine a video frame, in which the posture of the target object is detected as a second predetermined posture, to be the end frame.

12. The terminal of claim 8 , wherein the processor is further arranged to:
overlay an area at a target position in the background frame with the image area; and
feather edges of the image area with which the background frame was overlaid to form the special effect frame.

13. The terminal of claim 12 , wherein the processor is further arranged to:
determine an alignment parameter according to the position of the target object in each of the video frames; and
overlay, according to the alignment parameter, the area at the target position in the background frame with the image area of the target object.

14. The terminal of claim 13 , wherein the processor is further arranged to:
select a video frame from the video frames in the video as a reference frame; and
determine the alignment parameter according to a position offset of the target object in the video frame relative to the target object in the reference frame.

15. A non-transitory computer readable storage medium having computer executable instructions stored thereon, the computer executable instructions, when being executed by a processor, cause the processor to implement:
selecting, from video frames in a video to be processed, a background frame for a static special effect; and
for each of the video frames in the video, acquiring an image area, where a target object is located, from the respective video frame, and fusing the image area into the background frame, to generate a special effect frame with the static special effect,
wherein selecting, from the video frames in the video to be processed, the background frame for the static special effect comprises:
determining a first frame after the video to be processed enters a time static mode in which the static special effect is formed, as the background frame for the static special effect.

Assignees

• Beijing Xiaomi Pinecone Electronics Co Ltd

Inventors

• Chang Binglin
• Ma Kailun
• Shi Zhenghui
• Wang Qingmin

Classifications

• G06F18/25

  Fusion techniques · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title