Presenting information on similar objects relative to a target object from a plurality of video frames

Patent metadata

Field	Value
Publication number	US-11284168-B2
Application number	US-202017076345-A
Country	US
Kind code	B2
Filing date	Oct 21, 2020
Priority date	Oct 24, 2019
Publication date	Mar 22, 2022
Grant date	Mar 22, 2022

Abstract

Presenting information on similar objects relative to a target object is disclosed, including: obtaining a plurality of video frames; determining a target object in the plurality of video frames using a tracking recognition technique; determining time slice information corresponding to the target object; using the time slice information corresponding to the target object to determine one or more similar objects relative to the target object; receiving an indication to present information on the one or more similar objects relative to the target object; and outputting the information on the one or more similar objects relative to the target object.

First claim

What is claimed is:

1. A system, comprising:
one or more processors configured to:
obtain a plurality of video frames;
determine a target object in the plurality of video frames using a tracking recognition technique;
determine time slice information corresponding to the target object;
use the time slice information corresponding to the target object to determine one or more similar objects relative to the target object;
compare current playback information of a playing of the plurality of video frames to a time range defined by the time slice information corresponding to the target object, wherein the time range defined by the time slice information is associated with a segment of the plurality of video frames that includes an appearance of the target object;
determine that the current playback information is within the time range defined by the time slice information corresponding to the target object;
in response to the determination that the current playback information is within the time range defined by the time slice information corresponding to the target object, present information on the one or more similar objects relative to the target object; and
output the information on the one or more similar objects relative to the target object; and

one or more memories coupled to the one or more processors and configured to provide the one or more processors with instructions.

2. The system of claim 1 , wherein to use the time slice information corresponding to the target object to determine the one or more similar objects relative to the target object comprises to:
determine a set of video frames that includes the target object based at least in part on the time slice information corresponding to the target object; and
use the set of video frames to search in a similar object library for the one or more similar objects relative to the target object.

3. The system of claim 2 , wherein to use the set of video frames to search in the similar object library for the one or more similar objects relative to the target object comprises to:
determine a keyframe among the set of video frames that includes the target object; and
use the keyframe to search in the similar object library for the one or more similar objects relative to the target object.

4. The system of claim 3 , wherein to determine the keyframe among the set of video frames that includes the target object comprises to:
for each video frame in the set of video frames that includes the target object, determine a corresponding bounding box corresponding to the target object in that video frame; and
determine the keyframe based at least in part on a largest bounding box being associated with the target object in the keyframe.

5. The system of claim 3 , wherein the one or more processors are further configured to store a mapping between identifying information of the keyframe and the information on the one or more similar objects relative to the target object.

6. The system of claim 1 , wherein the one or more processors are further configured to store a mapping between identifying information of the target object and the information on the one or more similar objects relative to the target object.

7. The system of claim 1 , wherein the one or more processors are further configured to:
determine that a number of the one or more similar objects relative to the target object is greater than a preset number;
determine respective degrees of similarity between the one or more similar objects relative to the target object and the target object;
rank the one or more similar objects relative to the target object based on the respective degrees of similarity; and
select the preset number of the ranked one or more similar objects relative to the target object for which information is to be output.

8. The system of claim 1 , wherein to output the information on the one or more similar objects relative to the target object comprises to cause the information on the one or more similar objects relative to the target object to be presented as bullet screen messages in a manner that overlays the playing of the plurality of video frames.

9. A method, comprising:
obtaining a plurality of video frames;
determining a target object in the plurality of video frames using a tracking recognition technique;
determining time slice information corresponding to the target object;
using the time slice information corresponding to the target object to determine one or more similar objects relative to the target object;
comparing current playback information of a playing of the plurality of video frames to a time range defined by the time slice information corresponding to the target object, wherein the time range defined by the time slice information is associated with a segment of the plurality of video frames that includes an appearance of the target object;
determining that the current playback information is within the time range defined by the time slice information corresponding to the target object;
in response to the determination that the current playback information is within the time range defined by the time slice information corresponding to the target object, presenting information on the one or more similar objects relative to the target object; and
outputting the information on the one or more similar objects relative to the target object.

10. The method of claim 9 , wherein using the time slice information corresponding to the target object to determine the one or more similar objects relative to the target object comprises:
determining a set of video frames that includes the target object based at least in part on the time slice information corresponding to the target object; and
using the set of video frames to search in a similar object library for the one or more similar objects relative to the target object.

11. The method of claim 10 , wherein using the set of video frames to search in the similar object library for the one or more similar objects relative to the target object comprises:
determining a keyframe among the set of video frames that includes the target object; and
using the keyframe to search in the similar object library for the one or more similar objects relative to the target object.

12. The method of claim 11 , wherein determining the keyframe among the set of video frames that includes the target object comprises:
for each video frame in the set of video frames that includes the target object, determining a corresponding bounding box corresponding to the target object in that video frame; and
determining the keyframe based at least in part on a largest bounding box being associated with the target object in the keyframe.

13. The method of claim 11 , further comprising storing a mapping between identifying information of the keyframe and the information on the one or more similar objects relative to the target object.

14. The method of claim 9 , further comprising storing a mapping between identifying information of the target object and the information on the one or more similar objects relative to the target object.

15. The method of claim 9 , further comprising:
determining that a number of the one or more similar objects relative to the target object is greater than a preset number;
determining respective degrees of similarity between the one or more similar objects relative to the target object and the target object;
ranking the one or more similar objects relative to the target object based on the respective degrees of similarity; and
selecting the preset number of the ranked one or more similar

Assignees

• Alibaba Group Holding Ltd

Inventors

• Zhang Yanhao
• Xie Chenwei
• Shen Yuan
• Lin Jie
• Zheng Yun
• Pan Pan
• Xu Yinghui

Classifications

• H04N21/812 Primary

  involving advertisement data (advertising per se G06Q30/02) · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06V10/22

  by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition · CPC title

• H04N21/8133 Primary

  specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme · CPC title

• H04N21/4826

  using recommendation lists, e.g. of programmes or channels sorted out according to their score · CPC title