Online domain adaptation for multi-object tracking

Patent metadata

Field	Value
Publication number	US-9984315-B2
Application number	US-201514704350-A
Country	US
Kind code	B2
Filing date	May 5, 2015
Priority date	May 5, 2015
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

Methods and systems for online domain adaptation for multi-object tracking. Video of an area of interest can be captured with an image-capturing unit. The video (e.g., video images) can be analyzed with a pre-trained object detector utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure to jointly adapt online all trackers associated with the pre-trained object detector and a pre-trained category-level model from the trackers in order to efficiently track a plurality of objects in the video captured by the image-capturing unit.

First claim

The invention claimed is:

1. A method for online domain adaptation for multi-object tracking, said method comprising:
pre-training an object detector and a category-level model, wherein said pre-trained object detector is trained offline for at least one category of interest using a general-purpose labeled dataset and wherein said pre-trained object detector is associated with a plurality of trackers;
capturing video of an area of interest with a video camera; and
analyzing said video with said pre-trained object detector utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure, wherein said convex multi-task learning and said associated self-tuning stochastic optimization procedure jointly adapt online all trackers among said plurality of trackers associated with said pre-trained object detector and said pre-trained category-level model from said trackers to efficiently track a plurality of objects in said video captured by said video camera and wherein said associated self-tuning stochastic optimization procedure includes a use of learning rates and regularization parameters in which an update of at least one tracker of among said trackers includes a contribution of all other trackers among said trackers including both current and past trackers thereof, and wherein said learning rates are automatically set per-frame and per-target with respect to said video.

2. The method of claim 1 wherein said self-tuning stochastic optimization procedure comprises self-tuning of hyper-parameters.

3. The method of claim 2 further comprising selecting said hyper-parameters including at least two critical hyper-parameters comprising a learning rate among said learning rates and a regularization factor among said regularization parameters.

4. The method of claim 2 wherein said convex multi-task learning comprises logistic regression and hard negative mining.

5. The method of claim 1 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

6. The method of claim 5 wherein said convex multi-task learning comprises logistic regression and hard negative mining.

7. The method of claim 6 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

8. A system for online domain adaptation for multi-object tracking, said system comprising:
an image capturing unit that captures video of an area of interest; and
a pre-trained object detector that is pre-trained offline for at least one category of interest using a general-purpose labeled dataset and wherein said pre-trained object detector is associated with a plurality of trackers, wherein said pre-trained object detector analyzes said video utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure, wherein said convex multi-task learning and said associated self-tuning stochastic optimization procedure jointly adapt online all trackers among said plurality of trackers associated with said pre-trained object detector and a pre-trained category-level model from said trackers to efficiently track a plurality of objects in said video captured by said image capturing unit and wherein said associated self-tuning stochastic optimization procedure includes a use of learning rates and regularization parameters in which an update of at least one tracker of among said trackers includes a contribution of all other trackers among said trackers including both current and past trackers thereof, and wherein said learning rates are automatically set per-frame and per-target with respect to said video.

9. The system of claim 8 wherein said self-tuning stochastic optimization procedure comprises self-tuning of hyper-parameters.

10. The system of claim 9 wherein said convex multi-task learning comprises logistic regression and hard negative mining.

11. The system of claim 9 wherein said hyper-parameters include at least two critical hyper-parameters comprising a learning rate among said learning rates and a regularization factor from among said regularization parameters.

12. The system of claim 11 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

13. The system of claim 8 wherein said convex muti-task learning comprises logistic regression and hard negative mining.

14. The system of claim 13 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

15. A system for online domain adaptation for multi-object tracking, said system comprising:
at least one processor; and
a non-transitory computer-usable medium embodying computer program code, said computer-usable medium capable of communicating with said at least one processor, said computer program code comprising instructions executable by said at least one processor and configured for:
pre-training an object detector and a category-level model, wherein said pre-trained object detector is trained offline for at least one category of interest using a general-purpose labeled dataset and wherein said pre-trained object detector is associated with a plurality of trackers:
capturing video of an area of interest with a video camera; and
analyzing said video with said pre-trained object detector utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure, wherein said convex multi-task learning and said associated self-tuning stochastic optimization procedure jointly adapt online all trackers among said plurality of trackers associated with said pre-trained object detector and said pre-trained category-level model from said trackers to efficiently track a plurality of objects in said video captured by said video camera and wherein said associated self-tuning stochastic optimization procedure includes a use of learning rates and regularization parameters in which an update of at least one tracker of among said trackers includes a contribution of all other trackers among said trackers including both current and past trackers thereof, and wherein said learning rates are automatically set per-frame and per-target with respect to said video.

16. The system of claim 15 wherein said self-tuning stochastic optimization procedure comprises self-tuning of hyper-parameters.

17. The system of claim 16 wherein said instructions further comprise selecting said hyper-parameters including at least two critical hyper-parameters comprising a learning rate among said learning rates and a regularization factor from said regularization parameters.

18. The system of claim 17 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

19. The system of claim 16 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back, and wherein said convex multi-task learning comprises logistic regression and hard negative mining.

20. The system of claim 15 wherein said convex multi-task learning comprises logistic regression and hard negative mining.

Assignees

• Xerox Corp
• Condurent Business Services Llc

Inventors

• Gaidon Adrien
• Vig Eleonora

Classifications

• G06T7/251 Primary

  involving models · CPC title

• G06V30/194

  References adjustable by an adaptive method, e.g. learning · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06K9/66 Primary

  Physics · mapped topic

• G06T7/277

  involving stochastic approaches, e.g. using Kalman filters · CPC title