Blur object tracker using group lasso method and apparatus

Patent metadata

Field	Value
Publication number	US-2016125249-A1
Application number	US-201414528528-A
Country	US
Kind code	A1
Filing date	Oct 30, 2014
Priority date	Oct 30, 2014
Publication date	May 5, 2016
Grant date	—

Abstract

A method and apparatus for tracking an object across a plurality of sequential images, where certain of the images contain motion blur. A plurality of normal templates of a clear target object image and a plurality of blur templates of the target object are generated. In the next subsequent image frame, a plurality of bounding boxes are generated of potential object tracking positions about the target object location in the preceding image frame. For each bounding box image frame, a reconstruction error is generated that one bounding box has a maximum probability that it is the object tracking result in the subsequent image frame.

First claim

1 . A method for tracking a target object in sequential frames of images comprising:
determining a first tracking position associated with a target object in one frame of an image sequence; generating, by a processor, a plurality of normal templates about the first tracking position of the target object, and a plurality of blur templates; generating, by the processor, a plurality of bounding boxes about the first tracking position in a subsequent image frame of potential object tracking positions; combining a loss function and a l 1 +l 1 /l 2 if mixed norm; and generating, by the processor, for each of the plurality of bounding boxes surrounding a location of the target object in a subsequent image frame, a reconstruction error that one bounding box has a maximum probability that it is the target object tracking result.

2 . The method of claim 1 further comprising:
calculating a gradient histogram for the normal templates and the blur templates, and a gradient histogram of each bounding box.

3 . The method of claim 2 further comprising;
calculating a distance between the gradient histograms of each bounding box and the normal templates and the blur templates.

4 . (canceled)

5 . The method of claim 1 further comprising:
using a sum of l 1 +l 1 /l 2 mixed norms to regulate normal and blur templates coefficients.

6 . The method of claim 1 further comprising:
combining the loss function and the l 1 +l 1 /l 2 mixed norm to find a minimum reconstruction error for a target object in one frame image.

7 . A method for tracking objects comprising:
receiving an image sequence including a plurality of sequential image frames from an image sensor; selecting a target object in one image frame; segmenting the target object into a bounding box; creating a plurality of a normal templates about the bounding box; creating a plurality of blur templates of the bounding box by convolving the bounding box with different kernels; for the normal templates and the blur templates in one image frame, calculating a gradient histogram; in the next image frame, generating, by a processor, a plurality of target candidate bounding boxes about the bounding box in a preceding image frame; for each bounding box, calculating a gradient histogram and a distance of each target candidate bounding box gradient histogram and the gradient histograms of the normal templates and the blur templates; measuring a sum of weighted distance and a loss function; dividing the templates into separate groups of: normal templates, blur templates toward a same direction of motion, and trivial templates; using a structured sparsity-inducing norm that combines a sum of l 1 +l 1 /l 2 over groups of variables in each group of normal templates and blur templates; combining a loss function and the sum of l 1 +l 1 /l 2 mixed norms into a non-smooth convex optimization problem; and solving the non-smooth convex optimization problem to derive an observation likelihood from a reconstruction error of a location of the target object being tracked in a current image frame.

8 . An apparatus for tracking objects over a plurality of sequential image frames, where at least certain of the sequential image frames contain motion blur, the apparatus comprising:
an image sensor generating a plurality of sequential image frames of a target object in a field of view of the image sensor, the image sensor outputting the sequential image frames; and a processor configured to receive an image sequence including a plurality of sequential image frames from the image sensor, the processor executing program instructions to:
determine a first tracking position associated with a target object in a first image frame;
determine a plurality of normal templates about the first tracking position of the target object and a plurality of blur templates associated with the normal templates;
generate a plurality of bounding boxes in a subsequent frame of potential object tracking positions about the first tracking position in a preceding image frame;
combine a loss function and a l 1 +l 1 /l 2 mixed norm; and
generate a reconstruction error that one of the plurality of bounding boxes has a maximum probability that it is the object tracking position.

9 . The apparatus of claim 8 wherein the program instructions further comprise an instruction for:
calculating, by the processor, a gradient histogram for the normal and blurred templates.

10 . The apparatus of claim 9 wherein the program instructions further comprise an instruction for:
calculating, by the processor, a distance between a gradient histogram of a bounding box of the target object and the gradient histogram of normal templates and blur template.

11 . (canceled)

12 . The apparatus of claim 8 wherein the program instructions further comprise an instruction for:
using a sum of l 1 +l 1 /l 2 mixed norms to regulate normal and blur templates coefficients.

13 . The apparatus of claim 8 wherein the program instructions further comprise an instruction for:
combining, by the processor, a loss function and the l 1 +l 1 /l 2 mixed norm to find a minimum reconstruction error for a target candidate in one image frame.

14 . The apparatus of claim 8 wherein the program instructions further comprise instructions for:
selecting a target image in one image frame;
segmenting the target image into a target-bounding box;
creating the plurality of normal templates of an unblurred object in the target box;
creating a plurality of blur templates from the normal templates by convolving the normal templates with different kernels;
calculating a gradient histogram for the plurality of normal templates and the blur templates;
in a next image frame, generating a plurality of target candidate bounding boxes about a target object in a preceding image frame;
calculating a distance of each target candidate gradient histograms and the gradient histograms of the normal and blur templates;
measuring a sum of weighted distance and a loss function;
dividing the templates into separate groups of: normal templates, blur templates toward a same direction of motion, and trivial templates;
using a structured sparsity-inducing norm that combines a sum of l 1 +l 1 /l 2 over groups of variables in each group of normal templates and blur templates;
combining a loss function and the sum of l 1 +l 1 /l 2 mixed norm into a non-smooth convex optimization problem; and
solving the non-smooth convex optimization problem to derive an observation likelihood from a reconstruction error having a maximum probability that one bounding box of a location is the location of the object being tracked in a current image frame.

Assignees

• Toyota Eng & Mfg North America

Inventors

• Mei Xue
• Wu Yi
• Prokhorov Danil V
• Ling Haibin

Classifications

• G06T7/277 Primary

  involving stochastic approaches, e.g. using Kalman filters · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06F18/2136

  based on sparsity criteria, e.g. with an overcomplete basis · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title