Image processing method, apparatus, electronic device and computer readable storage medium

Patent metadata

Field	Value
Publication number	US-11651529-B2
Application number	US-202217843248-A
Country	US
Kind code	B2
Filing date	Jun 17, 2022
Priority date	Dec 18, 2019
Publication date	May 16, 2023
Grant date	May 16, 2023

Abstract

The embodiment of the present disclosure discloses an image processing method, apparatus, electronic device and computer readable storage medium. The image processing method includes identifying a first object in a first video frame image and a second object located in the first object; in accordance with a position of the first object in the first video frame image, overlaying a third object as a foreground image on the first video frame image to obtain a second video frame image; wherein the third object overlays the first object in the second video frame image; in accordance with a position of the second object in the first video frame image, overlapping the second object as a foreground image on the third object of the second video frame image to obtain a third video frame image.

First claim

What claimed is:

1. An image processing method, comprising:
identifying a first object in a first video frame image and a second object located in the first object;
in accordance with a position of the first object in the first video frame image, overlaying a third object as a foreground image on the first video frame image to obtain a second video frame image; wherein the third object overlays the first object in the second video frame image;
in accordance with a position of the second object in the first video frame image, overlapping the second object as a foreground image on the third object of the second video frame image to obtain a third video frame image,
wherein in accordance with the position of the second object in the first video frame image, overlapping the second object as a foreground image on the third object of the second video frame image to obtain a third video frame image comprises:
calculating a first distance between two key points preset on the first object;
acquiring a yaw angle and a pitch angle of the first object, wherein the yaw angle is a horizontal angle between an orientation of the first object in the first video frame image and a lens shooting direction, and the pitch angle is a vertical angle between the orientation of the first object in the first video frame image and the lens shooting direction;
calculating the offset of the second object based on the first distance, the yaw angle and the pitch angle; and
in accordance with the position of the second object in the first video frame image and the offset of the second object, overlapping the second object as a foreground image on the third object in the second video frame image to obtain the third video frame image.

2. The image processing method of claim 1 , wherein the identifying a first object in a first video frame image and a second object located in the first object comprises:
identifying a plurality of first key points of the first object and a plurality of second key points of the second object in the first video frame image;
identifying the first object based on the plurality of first key points of the first object;
identifying the second object based on the plurality of second key points of the second object, wherein the plurality of second key points are edge key points of the second object.

3. The image processing method of claim 2 , wherein in accordance with the position of the first object in the first video frame image, overlaying a third object as a foreground image on the first video frame image to obtain a second video frame image comprises:
calculating an anchor point of the first object in the first video frame image based on the plurality of first key points of the first object in the first video frame image;
by aligning an anchor point of the third object with the anchor point of the first object in the first video frame image, overlaying the third object as a foreground image in the first video frame image to obtain the second video frame image.

4. The image processing method of claim 1 , wherein in accordance with the position of the second object in the first video frame image and the offset of the second object, overlapping the second object as a foreground image on a third object in the second video frame image to obtain the third video frame image comprises:
performing position offset on an initial anchor point of the second object in the first video frame image in accordance with the offset of the second object to obtain a target anchor point;
in the second video frame image, rendering the second object as a foreground image to a position of the target anchor point to obtain the third video frame image.

5. The image processing method of claim 1 , further comprises, after obtaining the third video frame image:
converting color spaces of the second object and the third object in the third video frame image from a RGB color space to a HSV color space;
replacing a value of the second object in H channel in the HSV color space with a value of the third object in the H channel;
converting the color spaces of the second object and the third object from the HSV space to the RGB color space to obtain a fourth video frame image.

6. An electronic device, comprising:
a memory for storing computer readable instructions; and
a processor for executing the computer readable instructions, so that the computer readable instructions, when executed by the processor, cause the processor to:
identify a first object in a first video frame image and a second object located in the first object;
in accordance with a position of the first object in the first video frame image, overlay a third object as a foreground image on the first video frame image to obtain a second video frame image, wherein the third object overlays the first object in the second video frame image;
in accordance with a position of the second object in the first video frame image, overlap the second object as a foreground image on the third object of the second video frame image to obtain a third video frame image,
wherein the computer readable instructions include further instructions, when executed by the processor, cause the processor to, in accordance with the position of the second object in the first video frame image, overlap the second object as a foreground image on the third object of the second video frame image to obtain a third video frame image by:
calculating a first distance between two key points preset on the first object;
acquiring a yaw angle and a pitch angle of the first object, wherein the yaw angle is a horizontal angle between an orientation of the first object in the first video frame image and a lens shooting direction, and the pitch angle is a vertical angle between the orientation of the first object in the first video frame image and the lens shooting direction;
calculating the offset of the second object based on the first distance, the yaw angle and the pitch angle; and
in accordance with the position of the second object in the first video frame image and the offset of the second object, overlapping the second object as a foreground image on the third object in the second video frame image to obtain the third video frame image.

7. The electronic device of claim 6 , wherein the computer readable instructions include further instructions, when executed by the processor, cause the processor to identify a first object in a first video frame image and a second object located in the first object by:
identifying a plurality of first key points of the first object and a plurality of second key points of the second object in the first video frame image;
identifying the first object based on the plurality of first key points of the first object;
identifying the second object based on the plurality of second key points of the second object, wherein the plurality of second key points are edge key points of the second object.

8. The electronic device of claim 7 , wherein the computer readable instructions include further instructions, when executed by the processor, cause the processor to, in accordance with the position of the first object in the first video frame image, overlay a third object as a foreground image on the first video frame image to obtain a second video frame image by:
calculating an anchor point of the first object in the first video frame image based on the plurality of first key points of the first object in the first video frame image;
by aligning an anchor point of the third object with the anchor point of the first object in the first video frame image, overlaying the third object as a foreground image in the first video frame image to obtain the second video frame image.

9. The electronic device

Assignees

• Beijing Bytedance Network Tech Co Ltd

Inventors

• ZhuGe Jingjing
• Ni Guangyao
• Yang Hui

Classifications

• H04N5/272

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V40/165

  using facial parts and geometric relationships · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title