Tracking using multilevel representations

Patent metadata

Field	Value
Publication number	US-9519837-B2
Application number	US-201414323360-A
Country	US
Kind code	B2
Filing date	Jul 3, 2014
Priority date	Jul 3, 2014
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

A method of tracking a target object in frames of video data includes receiving a first tracking position associated with the target object in a first frame of a video sequence; identifying, for a second frame of the video sequence, a plurality of representation levels and at least one node for each representation level; determining, by a processor, a second tracking position in the second frame by estimating motion of the target object in the second frame between the first frame and the second frame; determining, at each representation level by the processor, a value for each node based on a conditional property of the node in the second frame; and adjusting, by the processor, the second tracking position based on the values determined for each of the nodes and interactions between at least some of the nodes at different representation levels.

First claim

What is claimed is:

1. A method of tracking a target object in frames of video data, comprising:
receiving a first tracking position associated with the target object in a first frame of a video sequence;
identifying, for a second frame of the video sequence, a plurality of representation levels, wherein the plurality of representation levels includes a bounding box level;
determining, by a processor, at least one node for each representation level, wherein the at least one node for the bounding box level is included in a second tracking position associated with the target object in the second frame determined by estimating motion of the target object between the first frame and the second frame;
determining, at each of the plurality of representation levels by the processor, a value for each node based on a conditional property of the node; and
adjusting, by the processor, the second tracking position based on the values determined for each of the nodes and interactions between at least the at least one node for the bounding box level and the at least one node for a different representation level.

2. The method of claim 1 , wherein the plurality of representation levels is hierarchical.

3. The method of claim 1 , wherein the plurality of representation levels comprises a pixel level, a superpixel level, and the bounding box level.

4. The method of claim 1 , wherein the first and second tracking positions each include a bounding box, with the bounding box included in the second tracking position being associated with the at least one node for the bounding box level.

5. The method of claim 1 , wherein the conditional property is a probability of a node label.

6. The method of claim 1 , wherein, at the bounding box level, the conditional property is a probability of a node pose.

7. The method of claim 1 , wherein the value for each node is determined as an energy potential value for the corresponding representation level in a Conditional Random Field model.

8. The method of claim 1 , wherein the interactions between the at least one node for the bounding box level and the at least one node for a different representation level are determined based on pairwise energy potential values therebetween in a Conditional Random Field model.

9. The method of claim 1 , wherein the plurality of representation levels comprises a pixel level, a superpixel level, and the bounding box level, and wherein the at least one node for each representation level is associated with one of a pixel, a superpixel, or a bounding box based on the corresponding representation level.

10. The method of claim 9 , further comprising:
determining, for training, a positive sample set and a negative sample set based on the first frame and the first tracking position;
determining a set of pixel level random fields and a set of superpixel level random fields based on training results from the positive sample set and the negative sample set; and
determining a set of superpixels by grouping pixels in the second frame based on the set of superpixel level random fields.

11. The method of claim 10 , further comprising:
determining if an occlusion exists within the adjusted second tracking position; and
updating the positive sample set and the negative sample set based on the determination that no occlusion exists within the adjusted second tracking position.

12. An apparatus for tracking a target object in frames of video data, comprising:
one or more processors; and
a memory for storing data and program instructions executed by the one or more processors, wherein the one or more processors are configured to executed instructions stored in the memory to:
receive a first tracking position associated with the target object in a first frame of a video sequence;
identify, for a second frame of the video sequence, a plurality of representation levels, wherein the plurality of representation levels includes a bounding box level;
determine at least one node for each representation level, wherein the at least one node for the bounding box level is included in a second tracking position associated with the target object in the second frame determined by estimating motion of the target object between the first frame and the second frame;
determine, at each of the plurality of representation levels, a value for each node based on a conditional property of the node; and
adjust the second tracking position based on the values determined for each of the nodes and interactions between at least the at least one node for the bounding box level and the at least one node for a different representation level.

13. The apparatus of claim 12 , wherein the plurality of representation levels is hierarchical.

14. The apparatus of claim 12 , wherein the plurality of representation levels comprises a pixel level, a superpixel level, and the bounding box level.

15. The apparatus of claim 12 , wherein the first and second tracking positions each include a bounding box, with the bounding box included in the second tracking position being associated with the at least one node for the bounding box level.

16. The apparatus of claim 12 , wherein, at the bounding box level, the conditional property is a probability of a node pose.

17. The apparatus of claim 12 , wherein the value for each node is determined as an energy potential value for the corresponding representation level in a Conditional Random Field model.

18. The apparatus of claim 12 , wherein the interactions between the at least one node for the bounding box level and the at least one node for a different representation level are determined based on pairwise energy potential values therebetween in a Conditional Random Field model.

19. The apparatus of claim 12 , wherein the second tracking position is adjusted based on interactions between at least some of the nodes at the same representation level.

20. The apparatus of claim 12 , wherein the instructions further comprise instructions stored in the memory to:
determine, for training, a positive sample set and a negative sample set based on the first frame and the first tracking position;
determine a set of pixel level random fields and a set of superpixel level random fields based on training results from the positive sample set and the negative sample set; and
determine a set of superpixels by grouping pixels in the second frame based on the set of superpixel level random fields.

Assignees

• Toyota Motor Eng & Mfg North America Inc
• Univ Of Tech Sydney

Inventors

• Mei Xue
• Wang Chaohui
• Hong Zhibin
• Prokhorov Danil V
• Tao Dacheng

Classifications

• G06T7/207 Primary

  for motion estimation over a hierarchy of resolutions (multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53) · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06K2009/3291

  Physics · mapped topic

• G06K9/3241 Primary

  Physics · mapped topic

• G06T7/2073

  Physics · mapped topic