Multimedia object tracking and merging

Patent metadata

Field	Value
Publication number	US-12482111-B2
Application number	US-202217877639-A
Country	US
Kind code	B2
Filing date	Jul 29, 2022
Priority date	Jul 29, 2022
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

In multimedia object tracking and merging of tracked objects, an object is tracked through frames of multimedia content until a frame appears in which the tracked object is not detected. A first track is designated as one or more consecutive frames in which the tracked object is detected, the first track ending at the first frame. Tracking continues to try to detect the tracked object in a second frame subsequent to the first frame. If the tracked object is not again detected, information about the first track is output. If the tracked object is detected subsequently, a second track of consecutive tracked object detection is designated. The tracked objects in the two tracks are then compared with the aid of trained data models, and a matching score is determined to reflect the degree of match. If the matching score meets or exceeds a first threshold, the compared tracks are merged using the same identifier assigned to both tracks. If the matching score does not exceed a second threshold that is less than the first threshold, the tracks may be discarded as showing no match. If the matching score falls between the first and second thresholds, an indication is output for further analysis of the compared tracked objects.

First claim

What is claimed is:

1 . One or more non-transitory computer-readable media storing computer-executable instructions that upon execution cause one or more processors to perform operations comprising:
tracking an object through a first plurality of frames of multimedia content until a first frame appears in which the tracked object is not detected; designating as a first track at least one consecutive frame of the first plurality of frames in which the tracked object is detected, the first track ending before the first frame and each frame of the first track including the tracked object; determining whether the tracked object is detected in a second frame subsequent to the first frame in which the tracked object is not detected; and performing either:
in response to determining that the tracked object is not detected in the second frame subsequent to the first frame, outputting information about the first track; or
in response to determining that the tracked object is detected in the second frame subsequent to the first frame:
tracking the tracked object through a second plurality of frames that follow the second frame until a third frame appears in which the tracked object is not detected;
designating as a second track the second plurality of frames in which the tracked object is detected, the second track including the second frame and ending before the third frame; and
outputting information about the first and second tracks, each frame of the first and second tracks including the tracked object.

2 . The one or more non-transitory computer-readable media of claim 1 , wherein the operations further comprise:
receiving tracked object data about the tracked objects in the first and second tracks; selecting one or more data models; comparing the tracked objects in the first and second tracks using the selected one or more data models; and outputting an indication for further analysis of compared tracked objects that fail to meet criteria defined by matching thresholds.

3 . The one or more non-transitory computer-readable media of claim 2 , wherein the operations further comprise:
determining a matching score based on the comparison of the tracked objects; if the matching score meets or exceeds a first threshold of the matching thresholds:
merging the compared tracks using the same identifier assigned to the compared tracks; and
adding the identifier to metadata of each of the compared tracks;
if the matching score does not exceed a second threshold of the matching thresholds that is less than the first threshold, outputting an indication that the tracked objects are not a match; and if the matching score falls between the first and second thresholds, outputting the indication for further analysis of the compared tracked objects.

4 . The one or more non-transitory computer-readable media of claim 3 , wherein:
the matching score represents a statistical likelihood that the compared tracked objects are the same object, matching scores that meet or exceed the first threshold are in a high tier, matching scores that do not exceed the second threshold are in a low tier, and matching scores that fall between the first and second thresholds are in a middle tier; and the operations further comprise automatically processing the tracked objects in the merged tracks.

5 . The one or more non-transitory computer-readable media of claim 4 , wherein processing the tracked objects in the merged tracks comprises:
redacting a feature of the tracked objects.

6 . The one or more non-transitory computer-readable media of claim 4 , wherein processing the tracked objects in the merged tracks comprises:
highlighting the tracked objects.

7 . The one or more non-transitory computer-readable media of claim 4 , wherein the low, middle, and high tiers are not all the same size.

8 . A computer-implemented method, comprising:
tracking an object through a first plurality of frames of multimedia content until a first frame appears in which the tracked object is not detected; designating as a first track at least one consecutive frame of the first plurality of frames in which the tracked object is detected, the first track ending before the first frame and each frame of the first track including the tracked object; determining whether the tracked object is detected in a second frame subsequent to the first frame in which the tracked object is not detected; and performing either:
in response to determining that the tracked object is not detected in the second frame subsequent to the first frame, outputting information about the first track; or
in response to determining that the tracked object is detected in the second frame subsequent to the first frame:
tracking the tracked object through a second plurality of frames that follow the second frame until a third frame appears in which the tracked object is not detected;
designating as a second track the second plurality of frames in which the tracked object is detected, the second track including the second frame and ending before the third frame; and
outputting information about the first and second tracks, each frame of the first and second tracks including the tracked object.

9 . The computer-implemented method of claim 8 , further comprising:
receiving tracked object data about the tracked objects in the first and second tracks; selecting one or more data models; comparing the tracked objects in the first and second tracks using the selected one or more data models; and outputting an indication for further analysis of compared tracked objects that fail to meet criteria defined by matching thresholds.

10 . The computer-implemented method of claim 9 , further comprising:
determining a matching score based on the comparison of the tracked objects; if the matching score meets or exceeds a first threshold of the matching thresholds:
merging the compared tracks using the same identifier assigned to the compared tracks; and
adding the identifier to metadata of each of the compared tracks;
if the matching score does not exceed a second threshold of the matching thresholds that is less than the first threshold, outputting an indication that the tracked objects are not a match; and if the matching score falls between the first and second thresholds, outputting the indication for further analysis of the compared tracked objects.

11 . The computer-implemented method of claim 10 , wherein:
the matching score represents a statistical likelihood that the compared tracked objects are the same object, matching scores that meet or exceed the first threshold are in a high tier, matching scores that do not exceed the second threshold are in a low tier, and matching scores that fall between the first and second thresholds are in a middle tier; and the method further comprises automatically processing the tracked objects in the merged tracks.

12 . The computer-implemented method of claim 11 , wherein processing the tracked objects in the merged tracks comprises:
redacting a feature of the tracked objects.

13 . The computer-implemented method of claim 11 , wherein processing the tracked objects in the merged tracks comprises:
highlighting the tracked objects.

14 . The computer-implemented method of claim 11 , wherein the low, middle, and high tiers are not all the same size.

15 . A computing device, comprising:
one or more processors; and a memory having one or more non-transitory computer-readable media that store computer-executable instructions that upon execution cause the one or more processors to perform opera

Assignees

• Getac Technology Corp
• Whp Workflow Solutions Inc

Inventors

• Adeel Muhammad
• Guzik Thomas

Classifications

• G06V10/74

  Image or video pattern matching; Proximity measures in feature spaces · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06V2201/10

  Recognition assisted with metadata · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title