Method for detecting and tracking a target, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-12511758-B2
Application number	US-202118040721-A
Country	US
Kind code	B2
Filing date	Aug 6, 2021
Priority date	Aug 6, 2020
Publication date	Dec 30, 2025
Grant date	Dec 30, 2025

Abstract

A method for detecting and tracking a target is provided. The method includes: acquiring a target tracking result of each video frame of a plurality of video frames received from a video stream by inputting the each video frame of the plurality of video frames into a tracking network; in response to a video frame of the plurality of video frames being a key frame, acquiring a target detection result output by a detection network upon receiving a last delay frame in a specified number of delay frames by inputting the key frame into the detection network and controlling the detection network to operate during a period of receiving the specified number of delay frames; and generating a final target tracking result based on the target detection result and a target tracking result of the last delay frame.

First claim

What is claimed is:

1 . A method for detecting and tracking a target, comprising:
acquiring a target tracking result of each video frame of a plurality of video frames received from a video stream by inputting the each video frame of the plurality of video frames into a tracking network, wherein the tracking network is configured to track a detected target in a video frame; wherein
in response to a video frame of the plurality of video frames being a key frame, acquiring a target detection result output by a detection network upon receiving a last delay frame in a specified number of delay frames by simultaneously inputting the key frame into the detection network and controlling the detection network to operate during a period of receiving the specified number of delay frames, wherein the key frame is a video frame for target detection in the video stream, and the specified number of delay frames are successive video frames following the key frame, wherein
the detection network comprises a plurality of computing nodes, the plurality of computing nodes being configured to control suspension or resumption of an operation of the detection network during the operation of the detection network; wherein
the detection network is a cascaded detection network, and the target detection result is acquired by: upon inputting the key frame into the detection network, inputting a received delay frame into the detection network and controlling the plurality of computing nodes of the detection network to operate sequentially in response to sequentially receiving the specified number of delay frames, and acquiring the target detection result output by the detection network upon receiving the last delay frame; or
the detection network is a non-cascaded detection network, and the target detection result is acquired by: upon inputting the key frame into the detection network, controlling the plurality of computing nodes of the detection network to operate sequentially in response to sequentially receiving the specified number of delay frames, and acquiring the target detection result output by the detection network upon receiving the last delay frame; and

generating a final target tracking result of the last delay frame based on the target detection result and a target tracking result of the last delay frame, wherein a tracked target list in the final target tracking result of the last delay frame is updated based on a detected target list in the target detection result, and target tracking for subsequent frames is performed based on the updated tracked target list.

2 . The method according to claim 1 , wherein acquiring the target detection result output by the detection network upon receiving the last delay frame in the specified number of delay frames by inputting the key frame into the detection network and controlling the detection network to operate during the period of receiving the specified number of delay frames comprises:
in the case that the detection network is the non-cascaded detection network, determining the specified number of delay frames and determining the plurality of computing nodes of the detection network based on the specified number; or, in the case that the detection network is the cascaded detection network, determining the plurality of computing nodes of the detection network and determining the specified number of delay frames based on the plurality of computing nodes of the detection network.

3 . The method according to claim 2 , wherein
the detection network is the cascaded detection network comprising multiple levels of sub-networks; and determining the plurality of computing nodes of the detection network comprises:
acquiring a first number of the sub-networks of the cascaded detection network; and
setting an input node of each of the sub-networks in the first number of the sub-networks as the computing node.

4 . The method according to claim 3 , wherein determining the specified number of delay frames based on the plurality of computing nodes of the detection network comprises:
determining, based on the first number, a plurality of successive video frames following the key frame as the specified number of delay frames.

5 . The method according to claim 4 , wherein controlling the plurality computing nodes of the detection network to operate sequentially in response to sequentially receiving the specified number of delay frames, and acquiring the target detection result output by the detection network upon receiving the last delay frame comprises:
upon inputting the key frame into a first-level sub-network of the cascaded detection network, each time one of the delay frames is received, inputting a previous detection result output by a previous-level sub-network and the one of the delay frames into a current-level sub-network; and upon receiving the last delay frame, determining a current detection result output by the current-level sub-network as the target detection result of the cascaded detection network.

6 . The method according to claim 2 , wherein
the detection network is a non-cascaded detection network; and determining the specified number of delay frames comprises:
determining a specified number of successive video frames following the key frame as the specified number of delay frames.

7 . The method according to claim 6 , wherein determining the plurality of computing nodes of the detection network based on the specified number comprises:
predicting a total duration required for the target detection on the key frame by the non-cascaded detection network; and calculating an operation suspend time of each of the computing nodes of the non-cascaded detection network based on the total duration and the specified number.

8 . The method according to claim 7 , wherein upon inputting the key frame into the detection network, controlling the plurality computing nodes of the detection network to operate sequentially in response to sequentially receiving the specified number of delay frames, and acquiring the target detection result output by the detection network upon receiving the last delay frame comprises:
timing an operation duration of the non-cascaded detection network upon inputting the key frame into the non-cascaded detection network; in response to the operation duration reaching the operation suspend time of the computing node, suspending an operation of the non-cascaded detection network; resuming the operation of the non-cascaded detection network from a suspended computing node each time one of the delay frames is received; and in response to the last delay frame being received, determining a detection result output, upon resumption of the operation of the non-cascaded detection network, by the non-cascaded detection network as the target detection result of the non-cascaded detection network.

9 . The method according claim 1 , wherein upon acquiring the target detection result output by the detection network upon receiving the last delay frame in the specified number of delay frames by inputting the key frame into the detection network and controlling the detection network to operate during the period of receiving the specified number of delay frames, the method further comprises:
determining whether the video frame is the key frame; and in response to the video frame being the key frame, performing the process of acquiring the target detection result output by the detection network upon receiving the last delay frame in the specified number of delay frames by inputting the key frame into the detection network and controlling the detection network to operate during the period of receiving the specified number of delay frames; and the method further compris

Assignees

• Bigo Tech Pte Ltd

Inventors

• Xiang Wei
• Wang Jundong
• Chen Lvran
• Jiao Jiening

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06T7/20 Primary

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title