Low- and high-fidelity classifiers applied to road-scene images

Patent metadata

Field	Value
Publication number	US-10373019-B2
Application number	US-201614995134-A
Country	US
Kind code	B2
Filing date	Jan 13, 2016
Priority date	Jan 13, 2016
Publication date	Aug 6, 2019
Grant date	Aug 6, 2019

Abstract

Disclosures herein teach applying a set of sections spanning a down-sampled version of an image of a road-scene to a low-fidelity classifier to determine a set of candidate sections for depicting one or more objects in a set of classes. The set of candidate sections of the down-sampled version may be mapped to a set of potential sectors in a high-fidelity version of the image. A high-fidelity classifier may be used to vet the set of potential sectors, determining the presence of one or more objects from the set of classes. The low-fidelity classifier may include a first Convolution Neural Network (CNN) trained on a first training set of down-sampled versions of cropped images of objects in the set of classes. Similarly, the high-fidelity classifier may include a second CNN trained on a second training set of high-fidelity versions of cropped images of objects in the set of classes.

First claim

The invention claimed is:

1. A method for object classification and location information detection, comprising:
down-sampling an image to a down-sampled version of the image; wherein down-sampling the image to the down-sampled version of the image comprises calculating a maximum factor by which the image can be down-sampled to generate the down-sampled version while maintaining a ratio of entropy in the down-sampled version to entropy in the image above a predetermined threshold level;
extracting a set of overlapping zones covering the down-sampled version, as definable by a sliding window with dimensions equal to dimensions of the set of overlapping zones;
selecting a probable zone from the set of overlapping zones for which a low-fidelity classifier, comprising a first Convolutional Neural Network (CNN), indicates a probability of a presence of an object pertaining to a class of objects classifiable by the low-fidelity classifier;
mapping the probable zone selected from the down-sampled version to a sector of a higher-resolution version of the image;
confirming the presence of the object by applying the sector to a high-fidelity classifier, comprising a second CNN, where applying the sector indicates the presence; and
providing a driving assistance to an automated driving system of a vehicle to be executed by the automated driving system based on the presence of the object.

2. The method of claim 1 , further comprising:
cropping a set of images of objects at a set of image sizes, images in the set of images classified according to a set of detection classes by labels assigned to the images;
down-sampling the set of images to create a down-sampled set of labeled images;
training the low-fidelity classifier with the down-sampled set of labeled images; and training the high-fidelity classifier with at least one of the set of images and comparable images selected for purposes of training.

3. The method of claim 2 , further comprising:
collecting a training set of images depicting pedestrians in various positions and contexts for inclusion within the set of images; and labeling the training set of images according to a common class in the set of detection classes.

4. The method of claim 1 , further comprising searching zones in the set of overlapping zones to which the low-fidelity classifier has yet to be applied for at least one additional probable zone while simultaneously confirming the presence of the object by applying the sector to the high-fidelity classifier.

5. The method of claim 1 , further comprising:
capturing, by a camera affixed to the vehicle, a series of images of oncoming road-scenes at a frame-rate satisfying a predefined threshold; and
processing the series of images at a processing-rate also satisfying the predefined threshold, the predefined threshold providing sufficient time for a pre-determined autonomous response by the automated driving system of the vehicle to classify information in the series of images.

6. The method of claim 1 , further comprising:
abstracting a set of scaled zones from the down-sampled version, scaled zones in the set of scaled zones having differing dimensions from the dimensions of the sliding window and commensurate with scaled dimensions of a scaled sliding window;
selecting a scaled zone from the set of scaled zones for which the low-fidelity classifier indicates a probability of an existence of a scaled object classifiable by the low-fidelity classifier;
mapping the scaled zone to a scaled sector of the higher-resolution version; and confirming the existence of the scaled object by applying the scaled sector to the high-fidelity classifier, where applying the scaled sector results in a probability of the existence.

Assignees

• Ford Global Tech Llc

Inventors

• Nariyambut Murali Vidya
• Schrier Madeline Jane

Classifications

• G06V10/87

  using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/285

  Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system · CPC title