Object-centric fine-grained image classification

Patent metadata

Field	Value
Publication number	US-9665802-B2
Application number	US-201514884963-A
Country	US
Kind code	B2
Filing date	Oct 16, 2015
Priority date	Nov 13, 2014
Publication date	May 30, 2017
Grant date	May 30, 2017

Abstract

Systems and methods are disclosed for classifying vehicles by performing scale aware detection; performing detection assisted sampling for convolutional neural network (CNN) training, and performing deep CNN fine-grained image classification to classify the vehicle type.

First claim

The invention claimed is:

1. A method for classifying a vehicle type, comprising:
performing scale aware detection;
performing detection assisted sampling for convolutional neural network (CNN) training; and
performing deep CNN fine grained image classification to classify the vehicle type;
wherein the scale aware detection is trained using a Regionlet detector, the Regionlet detector is a boosting classifier composed of weak classifiers:

H
⁡

(
x
)

=

∑

i
=
1

T

⁢

h
i

⁡

(
x
)

,

wherein T is the total number of training stages, h(x) is the weak classifier learned at stage t in training, x is the input image,
wherein the weak classifier h(x) is written as a function of the spatial location of Regionlets in h, and a feature used for h:
h t ( x )= G ( p t , f t , x ),

wherein p is a set of Regionlet locations, f is the feature extracted in the set of Regionlet locations.

2. The method of claim 1 , comprising using a selective search to generate object proposals for detection training and testing.

3. The method of claim 2 , wherein object proposals with more than 70% overlap with the ground truth are selected as positive samples during training and object proposals with less than 0.3% overlap with the ground truth are used as negative training samples.

4. The method of claim 1 , comprising apply multinomial sampling to images of the vehicle type.

5. The method of claim 1 , comprising applying regionlet re-localization method to learn a support vector regression model to predict an actual object location.

6. The method of claim 1 , comprising providing non-max suppression by taking the object proposal which gives the maximum detection response.

7. The method of claim 1 , comprising detecting an object with awareness of object scales and occlusions.

8. The method of claim 7 , wherein small detection responses are linked to small or occluded objects, or false alarms.

9. The method of claim 1 , comprising constructing a saliency aware dataset and using a scale aware object detection.

10. The method of claim 1 , comprising achieving occlusion awareness by training with visible objects.

11. The method of claim 1 , comprising labeling only a salient object in one image and checking consistency with a fine-grained category label.

12. The method of claim 1 , comprising labeling only one object as a detection ground truth for each image.

13. The method of claim 1 , comprising selecting an object based on mixed criteria of saliency.

14. The method of claim 13 , wherein the object is selected based on one or more of the following preferences: a big object preferred over small object, a visible object preferred over occluded object; a central object preferred over corner object, and consistency of an object's fine-grained category label with the image label.

15. A system to classify vehicles, comprising: a scale aware detector receiving an input image;
a deep convolutional neural network (CNN) coupled to the scale aware detector to classify a vehicle;
a detection assisted sampling module coupled to the scale aware detector, the sampling module generating data for CNN training; and
a deep CNN training module coupled to the detection assisted sampling module and the deep CNN;
wherein the scale aware detector is trained using a Regionlet detector, the Regionlet detector is a boosting classifier composed of weak classifiers:

H
⁡

(
x
)

=

∑

i
=
1

T

⁢

h
i

⁡

(
x
)

,

wherein T is the total number of training stages, h(x) is the weak classifier learned at stage tin training, x is the input image,
wherein the weak classifier h(x) is written as a function of the spatial location of Regionlets in h, and a feature used for h:
h t ( x )= G ( p t , f t , x ),

wherein p is a set of Regionlet locations, f is the feature extracted in the set of Regionlet locations.

16. The system of claim 15 , comprising a camera or a database of car images to provide input images.

17. The system of claim 15 , comprising a Regionlet detector for training the scale aware detection.

18. The system of claim 15 , comprising a vehicle or a machine controlled in part using the vehicle classification from the deep CNN.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Wang Xiaoyu
• Yang Tianbao
• Lin Yuanqing

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/2414

  Smoothing the distance, e.g. radial basis function networks [RBFN] · CPC title

• G06N3/045

  Combinations of networks · CPC title