Method and Apparatus for Detecting Region of Interest in Video, Device and Medium

Patent metadata

Field	Value
Publication number	US-2021383120-A1
Application number	US-202017116578-A
Country	US
Kind code	A1
Filing date	Dec 9, 2020
Priority date	Jun 5, 2020
Publication date	Dec 9, 2021
Grant date	—

Abstract

The present disclosure provides a method and apparatus for detecting a region of interest in a video, a device and a storage medium. The method may include: acquiring a current to-be-processed frame from a picture frame sequence of a video; detecting a region of interest (ROI) in the current to-be-processed frame, in response to determining that the current to-be-processed frame is a detection picture frame, to determine at least one ROI in the current to-be-processed frame; and updating a to-be-tracked ROI, based on the ROI in the current to-be-processed frame and a tracking result determined by a pre-order tracking picture frame; and tracking the current to-be-processed frame based on the existing to-be-tracked ROI, in response to determining that the current to-be-processed frame is a tracking picture frame, to determine at least one tracking result as the ROI of the current to-be-processed frame.

First claim

What is claimed is:

1 . A method for detecting a region of interest (ROI) in a video, the method comprising:
acquiring a current to-be-processed frame from a picture frame sequence of a video; detecting an ROI in the current to-be-processed frame, in response to determining that the current to-be-processed frame is a detection picture frame, to determine at least one ROI in the current to-be-processed frame; and updating a to-be-tracked ROI, based on the ROI in the current to-be-processed frame and a tracking result determined by a pre-order tracking picture frame; and tracking the current to-be-processed frame based on an existing to-be-tracked ROI, in response to determining that the current to-be-processed frame is a tracking picture frame, to determine at least one tracking result as the ROI of the current to-be-processed frame.

2 . The method according to claim 1 , wherein the updating a to-be-tracked ROI, based on the ROI in the current to-be-processed frame and a tracking result determined by a pre-order tracking picture frame, comprises:
matching the ROI in the current to-be-processed frame with an ROI in the tracking result of the pre-order tracking picture frame; and updating the to-be-tracked ROI, based on a matching result.

3 . The method according to claim 2 , wherein the updating the to-be-tracked ROI, based on a matching result, comprises:
adding the current ROI to the to-be-tracked ROI, in response to the current ROI in the current to-be-processed frame failing to match each ROI in the tracking result of the pre-order tracking picture frame; keeping the to-be-tracked ROI unchanged, in response to the current ROI in the current to-be-processed frame being successfully matched with any ROI in the tracking result of the pre-order tracking picture frame; and deleting the current ROI from the to-be-tracked ROI, in response to the current ROI in the tracking result of the pre-order tracking picture frame failing to match each ROI in the current to-be-processed frame.

4 . The method according to claim 2 , wherein the matching the ROI in the current to-be-processed frame with an ROI in the tracking result of the pre-order tracking picture frame, comprises:
determining an intersection over union of the ROI in the current to-be-processed frame and the ROI in the tracking result of the pre-order tracking picture frame; and determining a matching situation between each ROI in the current to-be-processed frame and each ROI in the tracking result of the pre-order tracking picture frame, based on each of the intersection over union.

5 . The method according to claim 1 , wherein the method further comprises:
determining a processing type of the current to-be-processed frame according to a detection and tracking strategy; and determining that the current to-be-processed frame is the detection picture frame or the tracking picture frame, based on the processing type; wherein, the processing type comprises a detection type and a tracking type.

6 . The method according to claim 5 , wherein the determining a processing type of the current to-be-processed frame according to a detection and tracking strategy, comprises:
determining that the processing type of the current to-be-processed frame is the detection type, in response to a frame interval between the current to-be-processed frame and a previous detection picture frame being a set number threshold; and determining that the processing type of the current to-be-processed frame is the tracking type, in response to the frame interval between the current to-be-processed frame and the previous detection picture frame being not the set number threshold.

7 . The method according to claim 5 , wherein the processing type further comprises a skip type; and after determining the processing type of the current to-be-processed frame according to the detection and tracking strategy, the method further comprises:
using a detection result of an ROI of a pre-order picture frame of the current to-be-processed frame as an ROI detection result of the current to-be-processed frame, in response to the current to-be-processed frame being of the skip type.

8 . The method according to claim 7 , wherein, the determining a processing type of the current to-be-processed frame according to the detection and tracking strategy, comprises:
determining a degree of difference between the pre-order picture frame of the current to-be-processed frame and the current to-be-processed frame; and determining that the processing type of the current to-be-processed frame is the skip type, in response to the degree of difference being less than a set difference degree threshold.

9 . The method according to claim 8 , wherein, the determining a degree of difference between the pre-order picture frame of the current to-be-processed frame and the current to-be-processed frame, comprises:
determining a histogram distance between the pre-order picture frame of the current to-be-processed frame and the current to-be-processed frame, and using the histogram distance as the degree of difference.

10 . The method according to claim 1 , wherein, after detecting or tracking the current to-be-processed frame, the method further comprises:
performing smoothing processing on the detection result or the tracking result of the ROI of the current to-be-processed frame, based on a detection result or a tracking result of an ROI of an adjacent picture frame of the current to-be-processed frame.

11 . The method according to claim 10 , wherein, the performing smoothing processing on the detection result or the tracking result of the ROI of the current to-be-processed frame, based on a detection result or a tracking result of an ROI of an adjacent picture frame of the current to-be-processed frame, comprises:
determining a weight of each adjacent picture frame, based on a frame spacing between the adjacent picture frame of the current to-be-processed frame and the current to-be-processed frame; and performing smoothing processing on the detection result or the tracking result of the ROI of the current to-be-processed frame, based on the weight of each adjacent picture frame and a detection result or a tracking result of an ROI of each adjacent picture frame.

12 . The method according to claim 1 , wherein the method further comprises:
performing validity verification on the tracking result of the pre-order tracking picture frame based on the ROI in the current to-be-processed frame, in response to the current to-be-processed frame being the detection picture frame.

13 . The method according to claim 12 , wherein, the performing validity verification on the tracking result of the pre-order tracking picture frame based on the ROI in the current to-be-processed frame, comprises:
matching the ROI in the current to-be-processed frame with an ROI in a tracking result of an adjacent historical tracking picture frame; acquiring a confidence of the tracking result of each of the pre-order tracking picture frame, in response to that the matching fails; and verifying the tracking result of each of the pre-order tracking picture frame, based on the confidence.

14 . The method according to claim 1 , wherein the method further comprises:
allocating different bite rates for the ROI and a non-ROI in the current to-be-processed frame; wherein a bite rate of the ROI is greater than a bite rate of the non-ROI; and encoding the current to-be-processed frame based on an allocated bite rate.

15 . An electronic device, comprising:
at least one processor; and a memory, communicatively con

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Zhou Zhichao
• He Dongliang
• Li Fu
• Sun Hao

Classifications

• G06T7/248 Primary

  involving reference images or patches · CPC title

• G06V20/40

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06T2207/20081

  Training; Learning · CPC title