Method and apparatus for displaying business object in video image and electronic device

Patent metadata

Field	Value
Publication number	US-11037348-B2
Application number	US-201715847172-A
Country	US
Kind code	B2
Filing date	Dec 19, 2017
Priority date	Aug 19, 2016
Publication date	Jun 15, 2021
Grant date	Jun 15, 2021

Abstract

Embodiments of the present disclosure provide a method and an apparatus for displaying a business object in a video image and an electronic device. The method for displaying a business object in a video image includes: detecting at least one target object from a video image, and determining a feature point of the at least one target object; determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object; and drawing the business object at the display position by using computer graphics. According to the embodiments of the present disclosure, the method and apparatus are conductive to saving network resources and system resources of a client.

First claim

What is claimed is:

1. A method for displaying a business object in a video image, comprising:
detecting at least one object from the video image, determining an object performing an action as a target object from the at least one object, and determining feature points of the target object by processing at least part of the video image, the features points of the target object being separated points on a contour of the target object;
determining a display position of a to-be-displayed business object in the video image according to the feature points on the contour of the target object by using a convolutional network model pre-trained by acquiring a feature vector of a business object sample image by the convolutional network model, the feature vector including information of the target object and position information and/or confidence information of a business object in the business object sample image; and
drawing the to-be-displayed business object at the display position by using computer graphics to display the to-be-displayed business object together with the target object, wherein the target object is a human body part performing the action, wherein the determining the display position of the to-be-displayed business object in the video image according to the feature points on the contour of the target object comprises: determining the human body part as the display position based on feature points on the contour of the human body part.

2. The method according to claim 1 , wherein pre-training of the convolutional network model further comprises:
performing convolution processing on the feature vector by using the convolutional network model to acquire a convolution result of the feature vector; and
adjusting a parameter of the convolutional network model according to the convolution result of the feature vector.

3. The method according to claim 2 , wherein the adjusting the parameter of the convolutional network model according to the convolution result of the feature vector comprises:
acquiring the corresponding position information of the business object in the convolution result of the feature vector; calculating a first distance between a position indicated by the corresponding position information of the business object and a preset standard position using a first loss function; and adjusting the parameter of the convolutional network model according to the first distance; and/or
acquiring the corresponding confidence information of the business object in the convolution result of the feature vector; calculating a second distance between a confidence indicated by the corresponding confidence information of the business object and a preset standard confidence using a second loss function; and adjusting the parameter of the convolutional network model according to the second distance.

4. The method according to claim 1 , wherein the determining a display position of a to-be-displayed business object in the video image according to the feature points on the contour of the target object comprises:
determining a type of the target object based on the feature points of the target object;
determining a display area of the to-be-displayed business object according to the type of the target object; and
determining the display position of the to-be-displayed business object in the video image according to the display area.

5. The method according to claim 1 , wherein the to-be-displayed business object comprises a plurality of associated business objects;
the determining a display position of a to-be-displayed business object in the video image comprises determining corresponding display positions of a plurality of to-be-displayed associated business objects in the video image; and
the drawing the to-be-displayed business object at the display position by using computer graphics comprises drawing the plurality of associated business objects at the corresponding display positions by using computer graphics, respectively.

6. The method according to claim 5 , wherein the plurality of associated business objects comprise any one or more of: multiple special effects containing semantic information that are used for displaying a same business object theme, multiple display portions of a same special effect containing semantic information, and multiple special effects containing semantic information that are provided by a same business object provider.

7. The method according to claim 1 , wherein the determining an object performing an action as a target object from the at least one object comprises: determining a second human body part with a gesture as the target object.

8. The method according to claim 1 , wherein the to-be-displayed business object has a same shape as the human body part.

9. The method according to claim 1 , wherein the to-be-displayed business object is an article for being placed on the human body part.

10. An apparatus for displaying a business object in a video image, comprising:
a processor; and
a memory storing instructions, the instructions when executed by the processor, cause the processor to perform operations, the operations comprising:
detecting at least one object from the video image, determining an object performing an action as a target object from the at least one object, and determining feature points of the target object by processing at least part of the video image, the features points of the target object being separated points on a contour of the target object;
determining a display position of a to-be-displayed business object in the video image according to the feature points on the contour of the target object by using a convolutional network model pre-trained by acquiring a feature vector of a business object sample image by the convolutional network model, the feature vector including information of the target object and position information and/or confidence information of a business object in the business object sample image; and
drawing the to-be-displayed business object at the display position by using computer graphics to display the to-be-displayed business object together with the target object, wherein the target object is a human body part performing the action, wherein the determining the display position of the to-be-displayed business object in the video image according to the feature points on the contour of the target object comprises: determining the human body part as the display position based on feature points on the contour of the human body part.

11. The apparatus according to claim 10 , wherein the operations further comprise pre-training the convolutional network model,
wherein the pre-training the convolutional network model comprises:
performing convolution processing on the feature vector to acquire a convolution result of the feature vector; and
adjusting a parameter of the convolutional network model according to the convolution result of the feature vector.

12. The apparatus according to claim 11 , wherein the adjusting the parameter of the convolutional network model according to the convolution result of the feature vector comprises:
acquiring the corresponding position information of the business object in the convolution result of the feature vector; calculating a first distance between a position indicated by the corresponding position information of the business object and a preset standard position using a first loss function; and adjusting the parameter of the convolutional network model according to the first distance; and/or
acquiring the corresponding confidence information of the business object in the convolution result of the feature vector; calculating a second distance

Assignees

• Beijing Sensetime Tech Development Co Ltd

Inventors

• Shi Jianping
• Luan Qing
• Xu Qinqin
• Wang Lei

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/23

  based on positionally close patterns or neighbourhood relationships · CPC title

• G06V10/225

  based on a marking or identifier characterising the area · CPC title

• G06N3/08

  Learning methods · CPC title