Effect video determination method and apparatus, and non-transitory storage medium

Patent metadata

Field	Value
Publication number	US-11736656-B1
Application number	US-202217880100-A
Country	US
Kind code	B1
Filing date	Aug 3, 2022
Priority date	May 17, 2022
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

Provided are an effect video determination method and apparatus, an electronic device and a storage medium. The method includes: acquiring effect operation information in a process of shooting a video, where the effect operation information includes at least one of a speech effect operation, a touch effect operation or a gesture effect operation; retrieving a target to-be-added effect corresponding to the effect operation information from an effect repository; fusing the target to-be-added effect and a to-be-processed video frame to determine a target effect video frame; and determining a target effect video based on a plurality of target effect video frames.

First claim

What is claimed is:

1. An effect video determination method, comprising:
acquiring effect operation information in a process of shooting a video, wherein the effect operation information comprises at least one of a speech effect operation, a touch effect operation or a gesture effect operation;
retrieving a target to-be-added effect corresponding to the effect operation information from an effect repository;
fusing the target to-be-added effect and a to-be-processed video frame to determine a target effect video frame; and
determining a target effect video based on a plurality of target effect video frames;
wherein before acquiring the effect operation information, the method further comprising:
displaying at least one to-be-selected effect operation mode on a display interface, and using a triggered to-be-selected effect operation mode as a target effect operation mode to acquire corresponding effect operation information based on the target effect operation mode such that effect operation information corresponding to an effect operation mode which is not triggered is not processed.

2. The method according to claim 1 , wherein determining triggering of shooting the video comprises at least one of the following:
detecting that a video shooting control is triggered;
detecting that a captured picture comprises a target object;
detecting that facial information is consistent with preset facial information;
detecting that speech information triggers a video shooting instruction; or
detecting that a body movement of a target object in a captured picture is relatively consistent with preset body movement information.

3. The method according to claim 1 , wherein when the effect operation information is the speech effect operation, retrieving the target to-be-added effect corresponding to the effect operation information from the effect repository comprises:
acquiring an audio data stream corresponding to the speech effect operation; and
retrieving the target to-be-added effect from the effect repository based on the audio data stream.

4. The method according to claim 3 , wherein retrieving the target to-be-added effect from the effect repository based on the audio data stream comprises:
retrieving the target to-be-added effect from the effect repository based on the audio data stream and a picture content in the to-be-processed video frame.

5. The method according to claim 4 , wherein the picture content comprises object information and current position information, and wherein retrieving the target to-be-added effect from the effect repository based on the audio data stream and the picture content in the to-be-processed video frame comprises:
retrieving the target to-be-added effect from the effect repository according to at least one keyword corresponding to the audio data stream and object information of a target object and current position information of the target object in the picture content, wherein the object information comprises object basic information, and the current position information comprises at least one of a scene position where the target object is located or a current geographical position of the target object.

6. The method according to claim 4 , wherein the picture content comprises object information and current position information, and wherein retrieving the target to-be-added effect from the effect repository based on the audio data stream and the picture content in the to-be-processed video frame comprises:
in a case where at least one keyword of the audio data stream comprises object name information, and the picture content comprises at least one target object, determining a target effect adding object corresponding to the object name information; and
retrieving the target to-be-added effect from the effect repository based on the at least one keyword and the target effect adding object.

7. The method according to claim 1 , wherein retrieving the target to-be-added effect corresponding to the effect operation information from the effect repository comprises:
determining the target to-be-added effect from the effect repository based on the effect operation information and a picture content in the to-be-processed video frame.

8. The method according to claim 7 , wherein when the effect operation information is the touch effect operation, determining the target to-be-added effect from the effect repository based on the effect operation information and the picture content in the to-be-processed video frame comprises:
acquiring the picture content in the to-be-processed video frame based on a triggering operation on a display interface, wherein the triggering operation corresponds to the touch effect operation; and
determining the target to-be-added effect according to a visual element in the picture content.

9. The method according to claim 8 , wherein determining the target to-be-added effect according to the visual element in the picture content comprises:
determining the target to-be-added effect according to the visual element in the picture content and position information corresponding to a touch point.

10. The method according to claim 1 , wherein when the effect operation information is the gesture effect operation, retrieving the target to-be-added effect corresponding to the effect operation information from the effect repository comprises:
detecting a gesture effect operation in a display interface to determine a target gesture pose; and
retrieving the target to-be-added effect from the effect repository according to the target gesture pose.

11. The method according to claim 10 , wherein retrieving the target to-be-added effect from the effect repository according to the target gesture pose comprises:
retrieving the target to-be-added effect from the effect repository according to the target gesture pose, position information of the target gesture pose, at least one display object corresponding to a picture content in the to-be-processed video frame and scene information of a scene to which the at least one display object belongs.

12. The method according to claim 1 , wherein when the effect operation information comprises the speech effect operation and the touch effect operation, retrieving the target to-be-added effect corresponding to the effect operation information from the effect repository comprises:
determining a to-be-added effect from the effect repository based on an audio data stream corresponding to the speech effect operation; and
processing, based on a picture content corresponding to a touch point of the touch effect operation on a display interface, the to-be-added effect to determine the target to-be-added effect;
wherein the picture content comprises at least one visual element, and the at least one visual element comprises at least one of a target object element, an environment element or an item element.

13. The method according to claim 1 , wherein when the effect operation information comprises the speech effect operation and the gesture effect operation, retrieving the target to-be-added effect corresponding to the effect operation information from the effect repository comprises:
determining a to-be-added effect from the effect repository based on an audio data stream corresponding to the speech effect operation; and
determining a target adding position of the to-be-added effect based on gesture position information corresponding to the gesture effect operation and a picture content, and processing the to-be-added effect based on the target adding position to obtain the target to-be-added effect.

14. The method according to claim 1 , wherei

Assignees

• Lemon Inc

Inventors

• Jiang Long
• Pan Keting
• Northway Ryan

Classifications

• H04N5/265

  Mixing · CPC title

• H04N5/2621 Primary

  Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability · CPC title

• H04N5/272

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title