System and method for re-identifying target object based on location information of CCTV and movement information of object

Patent metadata

Field	Value
Publication number	US-11527000-B2
Application number	US-202017038093-A
Country	US
Kind code	B2
Filing date	Sep 30, 2020
Priority date	Jan 22, 2020
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

Embodiments relate to a method for re-identifying a target object based on location information of closed-circuit television (CCTV) and movement information of the target object and a system for performing the same, the method including detecting at least one object of interest in a plurality of source videos based on a preset condition of the object of interest, tracking the identified object of interest on the corresponding source video to generate a tube of the object of interest, receiving an image query including a target patch and location information of the CCTV, determining at least one search candidate area based on the location information of the CCTV and the movement information of the target object, re-identifying if the object of interest seen in the tube of the object of interest is the target object, and providing a user with the tube of the re-identified object of interest.

First claim

The invention claimed is:

1. A method for re-identifying a target object based on location information of closes-circuit television (CCTV) and movement information of the target object, performed by a computing device including a processor, the method comprising:
receiving a plurality of source videos captured in a plurality of areas, wherein the plurality of areas includes areas in which at least one of a plurality of CCTVs are installed;
detecting at least one object of interest in the plurality of source videos based on a preset condition of the object of interest;
tracking the detected at least one object of interest on the corresponding source video to generate, for each detected object of interest, a tube of the respective object of interest, wherein generating each tube comprises:
when 1 st to n th source videos, in which a same first object of interest is seen in at least some frames, are received from 1 st to n th CCTVs installed in a same area respectively, wherein n is an integer of two or greater;
tracking the first object of interest in the 1 st to n th source videos to generate n tubes of the object of interest including #1-1 tube of the object of interest and #1-n tube of the object of interest; and
unifying the n tubes of the object of interest into the first object of interest;

receiving an image query including a target patch in which a target object is seen, and location information of the CCTV having captured the target object;
determining at least one search candidate area based on the location information of the CCTV having captured the target object and the movement information of the target object;
re-identifying if the object of interest seen in the tube of the object of interest is the target object to search for the tube of the object of interest in which the target object is seen, wherein the tube of the object of interest, on which the re-identification is performed, is the tube generated by capturing at least part of the determined search candidate area; and
providing a user with the tube of the re-identified object of interest as a query result.

2. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 1 , wherein detecting the at least one object of interest comprises:
detecting an object satisfying the condition of the object of interest including at least one of an object condition or an event condition in the plurality of source videos; and
identifying the detected object as the object of interest,
wherein the object condition includes an object type, and the event condition includes presence or absence of movement of the object.

3. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 1 , wherein generating the tube of the at least one object of interest comprises:
arranging a tracked frame including a patch of the object of interest tracked in a single source video in a time series manner, to generate the tube of the identified object of interest from the corresponding source video.

4. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 3 , wherein generating the tube of the at least one object of interest further comprises:
when a tracking failed frame in which tracking of the object of interest failed in the corresponding source video is acquired, recovering the tracking failed frame based on a first tracked frame captured in an earlier time than the tracking failed frame and a second tracked frame captured in a later time than the tracking failed frame; and
further arranging the first tracked frame, the recovered tracking failed frame, and the second tracked frame in a time series manner to generate the tube of the object of interest.

5. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 4 , wherein recovering the tracking failed frame comprises acquiring location information of the object of interest for which tracking failed on the tracking failed frame,
the location information of the object of interest on the tracking failed frame, acquired through the recovery of the tracking failed frame, includes at least one of a horizontal coordinate-vertical coordinate pair of the patch of the object of interest to be tracked, or a height-width pair of the patch of the object of interest, and
when the tracking failed frame is captured at t, a previous tracked frame is captured at t+a, and a subsequent tracked frame is captured at t+b,
each of the horizontal coordinate (x t ) and the vertical coordinate (y t ) of the patch of the object of interest in the tracking failed frame is calculated by the following Equation:
x t =|x t+b −x t+a |(( t+b )−( t+a ))

y t =|y t+b −y t+a |/(( t+b )−( t+a ))  [Equation]

each of the height (h t ) and the width (w t ) of the patch of the object of interest in the tracking failed frame is calculated by the following Equation:
w t =w t+b −w t+a |(( t+b )−( t+a ))

h t =|h t+b −h t+a |(( t+b )−( t+a ))  [Equation]

where x t+a , y t+a , w t+a , h t+a denote the horizontal coordinate, the vertical coordinate, the area and the depth of the patch of the object of interest tracked in the previous tracked frame respectively, and x t+b , y t+b , w t+b , h t+b denote the horizontal coordinate, the vertical coordinate, the area and the depth of the patch of the object of interest tracked in the subsequent tracked frame respectively.

6. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 1 , wherein the unifying comprises:
selecting at least one representative image for each of the n tubes of the object of interest in each tube; and
unifying the n tubes of the object of interest by clustering the tubes based on attribute information of the selected representative image.

7. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 6 , wherein the representative image of the tube is a patch of at least part of a frame having a smallest change in direction of the object of interest in the tube or a patch of at least part of a frame having a smallest change in speed of the object of interest in the tube.

8. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 6 , wherein the representative image of the tube is selected by calculating an image quality score for each of a plurality of patches of the object of interest included in the tube, and the image quality score is calculated based on detection accuracy of the object of interest, a degree of occlusion of the object of interest, and a distance between the object of interest and the CCTV.

9. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 1 , wherein determining the search candidate area comprises determining at least one area as the search candidate area based on direction and speed of the target patch.

10. The method for re-identifying a target object based on location information of CCTV and movement information of the target object according to claim 9 , wherein determining the search candidate area comprises determining the search candidate area by searching for a

Assignees

• Korea Inst Sci & Tech

Inventors

• Kim Ig Jae
• Choi Heeseung
• Kim Haksub
• Chae Seungho
• Yang Yoonsik

Classifications

• G06V10/993 Primary

  Evaluation of the quality of the acquired pattern · CPC title

• G06V10/772

  Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries · CPC title

• G06T7/292 Primary

  Multi-camera tracking · CPC title

• G06F16/7837 Primary

  using objects detected or recognised in the video content · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title