System and method for recognizing speed limit sign using front camera

Patent metadata

Field	Value
Publication number	US-9311543-B2
Application number	US-201414267338-A
Country	US
Kind code	B2
Filing date	May 1, 2014
Priority date	Jan 17, 2014
Publication date	Apr 12, 2016
Grant date	Apr 12, 2016

Abstract

Disclosed herein are a speed limit sign recognition system and method using a front camera which can track a traffic sign continuously appearing in images which are obtained by a photographing speed limit sign located in front of a driver using a camera mounted on the front of the vehicle, recognize an internal numeral of the traffic sign, and inform the driver of an actual limit speed through the recognized numeral, including: an image acquisition unit for acquiring a front image using a front camera; a traffic sign detector for detecting a traffic sign from the acquired front image; a recognition unit for recognizing an internal numeral in the detected traffic sign; a tracking unit for tracking a traffic sign continuously appearing in the front image, and eliminating a temporarily misrecognized object; and a decision unit for determining a result of speed limit recognition on the recognized traffic sign.

First claim

What is claimed is:

1. A method for recognizing a speed limit sign using a front camera, comprising:
acquiring a front image using a front camera;
detecting, by a traffic sign detector comprising a horizontal detector and a vertical detector, a traffic sign from the acquired front image using a Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm;
recognizing an internal numeral in the detected traffic sign;
tracking a traffic sign continuously appearing in the front image, and eliminating a temporarily misrecognized object; and
determining a result of recognition on the recognized traffic sign,
wherein a region of overlap of the horizontal detector and the vertical detector comprises an actual traffic sign region.

2. The method according to claim 1 , wherein, the Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm, which is used for object detection, is used to detect the traffic sign.

3. The method according to claim 2 , wherein the Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm is configured to include a plurality of strong classifiers in a cascade structure, and to identify whether an input image is a traffic sign image (positive sample) obtained by adding one or two margin pixels to an extracted image and then performing normalization into 20×20, or a non-traffic sign image (negative sample) randomly extracted from an image which does not include an image of the traffic sign.

4. The method according to claim 2 , wherein the Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm comprises:
a traffic sign training (sample training) step of training a detector using a traffic sign image (positive sample), which is an object to be actually recognized, and a non-traffic sign image (negative sample); and
a scan-window search step of determining whether a sub-window, which is received using a scan-window search scheme with respect to a region of interest (ROI) of the front image, corresponds to the actual traffic sign or a non-traffic sign.

5. The method according to claim 4 , wherein the Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm is configured to train the detector using the traffic sign image (positive sample) and the non-traffic sign image (negative sample) which are in a ratio of 1:2 in number.

6. The method according to claim 4 , wherein the Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm is configured to train for a part constituted by 20 horizontal and 10 vertical in an image normalized into 20×20 through the horizontal detector, or to train for a part constituted by 10 horizontal and 20 vertical in an image normalized into 20×20 through the vertical detector.

7. The method according to claim 6 , wherein the Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm is to detect the region on which the overlap occurs by the horizontal detector and the vertical detector, as the actual traffic sign region.

8. The method according to claim 1 , wherein, in the recognizing the internal numeral in the detected traffic sign, an actual limit speed is calculated using a support vector machine (SVM) with respect to a region which is determined as a traffic sign region in a detection result of the detecting the traffic sign.

9. The method according to claim 8 , wherein the recognizing the internal numeral in the detected traffic sign comprises:
a sample training step of detecting feature points using a traffic sign image (positive sample) and a non-traffic sign image (negative sample), training a recognizer with the feature points, and calculating a support vector; and
a speed recognition step of calculating probabilities of belonging to traffic sign image categories with inner products between the calculated support vector and feature points of an input image, and recognizing a numeral of the greatest probability value as a speed.

10. The method according to claim 1 , wherein, in the tracking the traffic sign, the traffic sign is tracked according to template matching, the inside of a traffic sign recognized in an image of time “t” is set as a template, a region of interest (ROI) in an image of time “t+1” is limited on the basis of a moving speed of a corresponding vehicle, a tracker is updated when a traffic sign exists in the limited region of interest, and a previous state is maintained when a traffic sign does not exists in the limited region of interest.

11. The method according to claim 1 , wherein, in the recognizing the internal numeral in the detected traffic sign,
the traffic sign being currently recognized is determined to have passed and tracking is terminated when the traffic sign has not been tracked during three or more frames, probability values of the respective traffic signs which are outputted as the results of an SVM with respect to tracking-terminated objects are accumulated, and a traffic sign having the greatest probability value is determined as a final result.

12. A system for recognizing a speed limit sign, comprising:
an image acquisition unit for acquiring a front image using a front camera;
a traffic sign detector for detecting a traffic sign from the acquired front image using a Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm, which is used for object detection;
a recognition unit for recognizing an internal numeral in the detected traffic sign;
a tracking unit for tracking a traffic sign continuously appearing in the front image, and eliminating a temporarily misrecognized object; and
a decision unit for determining a result of speed limit recognition on the recognized traffic sign,
wherein the traffic sign detector comprises a horizontal detector and a vertical detector, and
wherein a region of overlap of the horizontal detector and the vertical detector comprises an actual traffic sign region.

13. The system according to claim 12 , wherein the Haar-Feature based Viola-Jones Adaptive Boosting (Adaboost) algorithm is configured to include a plurality of strong classifiers in a cascade structure and to comprise:
a traffic sign training (sample training) step of training the traffic sign detector using a traffic sign image (positive sample), which is an object to be actually recognized, and a non-traffic sign image (negative sample); and
a scan-window search step of determining whether a sub-window, which is received using a scan-window search scheme with respect to a region of interest (ROI) of the front image, corresponds to the actual traffic sign or a non-traffic sign.

14. The system according to claim 13 , wherein the traffic sign detector is trained with the traffic sign image (positive sample) obtained by adding one or two margin pixels to an extracted image and then performing normalization into 20×20, and a non-traffic sign image (negative sample) randomly extracted from an image which does not include a traffic sign, the traffic sign image and the non-traffic sign image being in a ratio of 1:2 in number.

15. The system according to claim 12 , wherein the horizontal detector has been trained for a part constituted by 20 horizontal and 10 vertical in an image normalized into 20×20; and the vertical detector which has been trained for a part constituted by 10 horizontal and 20 vertical in an image normalized into 20×20.

16. The system according to claim 12 , wherein the recognition unit calculates an actual limit speed using a support vector machine (SVM) with respect to a region which is determined as a traffic sign region in a detection result of the traffic sign detector, wherein the recognition

Assignees

• Mando Corp

Inventors

• Cho Young Ha
• Lim Chae Seok
• Park Min Woo

Classifications

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V20/582

  of traffic signs · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06K9/00818 Primary

  Physics · mapped topic