Video object detection and tracking method and apparatus

Patent metadata

Field	Value
Publication number	US-12272137-B2
Application number	US-202117537762-A
Country	US
Kind code	B2
Filing date	Nov 30, 2021
Priority date	Dec 4, 2020
Publication date	Apr 8, 2025
Grant date	Apr 8, 2025

Abstract

A target object detection method and apparatus are provided. The target object detection method and apparatus are applicable to fields such as artificial intelligence, object tracking, object detection, and image processing. An object is detected from a frame image of a video including a plurality of frame images based on a target template set including one or more target templates.

First claim

What is claimed is:

1. An object detecting method, comprising:
detecting an object from a frame image of a video comprising a plurality of sequential frame images based on a target template set and an interference template set;
determining an integrated target feature by integrating image features of image areas respectively corresponding to target templates included in the target template set;
updating the target template set by adjusting the integrated target feature according to a change in similarity of the detected object across the sequential frame images; and
outputting detected object information regarding the detected object,
wherein the target template set comprises one or more target templates,
wherein each of the one or more target templates comprises respective frame object information of the object in respective frame images determined to be frame images including the object among other frame images previous to the sequential frame images,
wherein the interference template set comprises one or more interference templates,
wherein the detecting of the object from the frame image comprises:
obtaining one or more target candidate areas determined to be target candidate areas including the object in the frame image, based on the target template set and the interference template set; and
determining one target area from among the one or more target candidate areas, and

wherein the determining of the one target area from among the one or more target candidate areas comprises:
calculating matching degrees of the one or more target candidate areas to each target template of the target template set;
calculating matching degrees of the one or more target candidate areas to each interference template of the interference template set; and
calculating target matching degrees of the target candidate areas based on the matching degrees of the target candidate areas to each target template of the target template set and the matching degrees of the target candidate areas to each interference template of the interference template set.

2. The method of claim 1 , wherein the target template set further comprises an initial target template, and
wherein the initial target template comprises initial object information of the object in a frame image determined by a user to be a frame image that includes the object among the sequential frame images, or a separate image that is independent of the video and includes the object.

3. The method of claim 1 , wherein the
obtaining one or more target candidate areas is further based on the integrated target feature.

4. The method of claim 3 , wherein the obtaining of the one or more target candidate areas comprises:
determining a plurality of search areas within the frame image;
obtaining search area features by extracting respective image features of the plurality of search areas;
calculating correlations between the search area features of the plurality of search areas and the integrated target feature; and
determining the one or more target candidate areas from among the plurality of search areas based on the correlations.

5. The method of claim 3 , further comprising:
wherein the updating of the target template set comprises:
calculating the similarity of the target area to the integrated target feature of the target template set; and
adding the target area as a target template to the target template set in response to the similarity being less than a threshold.

6. The method of claim 3 , wherein the detecting of the object from the frame image comprises:
obtaining one or more target candidate areas determined to be target candidate areas including the object in the frame image; and
determining one target area from among the one or more target candidate areas,
wherein the method further comprises:
updating the target template set in response to the target area satisfying a predetermined condition.

7. The method of claim 6 , wherein the updating of the target template set comprises:
calculating similarities of the target area to all target templates of the target template set; and
adding the target area as a target template to the target template set in response to all the similarities of the target area being less than a threshold.

8. The method of claim 1 , wherein each of the one or more interference templates comprises interference object information regarding an interference object that interferes with the detection of the object among the other frame images.

9. The method of claim 1 , wherein the determining of the one target area from among the one or more target candidate areas comprises:
determining one target area from among the one or more target candidate areas based on the target matching degrees of the target candidate areas.

10. The method of claim 9 , wherein the calculating of the target matching degrees of the target candidate areas comprises calculating the target matching degrees of the target candidate areas based on one of a mean value and a median value of the matching degrees of the target candidate areas to each target template of the target template set and/or one of a mean value and a median value of the matching degrees of the target candidate areas to each interference template of the interference template set.

11. The method of claim 1 , wherein the obtaining of the one or more target candidate areas comprises:
determining the integrated target feature based on the target template set;
determining an integrated interference feature based on the interference template set; and
obtaining the one or more target candidate areas from the frame image based on the integrated target feature and the integrated interference feature.

12. The method of claim 11 , wherein the determining of the integrated target feature based on the target template set comprises determining the integrated target feature by integrating image features of image areas respectively corresponding to target templates included in the target template set, and
wherein the determining of the integrated interference feature based on the interference template set comprises determining the integrated interference feature by integrating image features of image areas respectively corresponding to interference templates included in the interference template set.

13. The method of claim 11 , wherein the determining of the integrated target feature based on the target template set comprises determining the integrated target feature based on all target templates included in the target template set, and
wherein the determining of the integrated interference feature based on the interference template set comprises determining the integrated interference feature based on all interference templates included in the interference template set.

14. The method of claim 11 , further comprising:
updating the interference template set,
wherein the updating of the interference template set comprises adding one of a portion or all of the other target candidate areas except for the one target area among the one or more target candidate areas as interference templates to the interference template set.

15. The method of claim 11 , further comprising:
updating the interference template set,
wherein the updating of the interference template set comprises:
calculating similarities of one of a portion or all of the other target candidate areas to the integrated interference feature of the interference template set; and
adding a target candidate area having a similarity less than a threshold value among the portion or all of the other target candidate areas a

Assignees

• Samsung Electronics Co Ltd

Inventors

• Xu Jingtao
• Chen Yiwei
• Park Changbeom
• Lee Hyunjeong
• Yoo Byung In
• Han Jaejoon
• Wang Qiang
• Yu Jiaqian

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/761 Primary

  Proximity, similarity or dissimilarity measures · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06V10/75

  Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries · CPC title

• G06V2201/07

  Target detection · CPC title