Method for detecting traffic violation

Patent metadata

Field	Value
Publication number	US-9704060-B2
Application number	US-201314775628-A
Country	US
Kind code	B2
Filing date	Dec 25, 2013
Priority date	Dec 17, 2013
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

The present invention relates to a technical field of traffic monitoring, and more particularly to a method for detecting traffic violation. The present invention includes firstly localizing vehicle salient parts through salient features including vehicle license numbers and vehicle rear lights, and representing a vehicle with the vehicle salient parts, then tracking the vehicle with a Kalman filter based on the vehicle salient parts, and finally detecting vehicle violation through moving trajectory analysis and setting violating detecting areas. The present invention solves vehicle violation detection problems in complex engineering application conditions such as illumination change and detection noise, and is suitable for city traffic management under complex conditions.

First claim

What is claimed is:

1. A method for detecting traffic violation, comprising steps of:
S 1 : calibrating a road traffic scene by a monitoring camera;
S 2 : localizing all vehicle license plates in a video sequence with an industrial personal computer;
S 3 : localizing all vehicle rear lights in the video sequence with the industrial personal computer;
S 4 : tracking vehicles based on a plurality of vehicle salient parts by the monitoring camera, for obtaining moving trajectories of the vehicles:
wherein according to a vehicle localizing result in a previous frame, a vehicle position in a current frame is predicted; a vehicle position result in the current frame is handled as a measured value, for searching an observed value of each of the vehicles tracked through calculating an Euclidean distance between a predicted position and a measured position; a predicted value and the observed value are calculated by weighting for updating the predicted value as a current position of an object; and
S 5 : analyzing the moving trajectories of the vehicles with the industrial personal computer, and setting detecting areas of different violations, so as to complete vehicle violation detection.

2. The method, as recited in claim 1 , wherein in the step S 4 , based on the vehicle salient parts, the vehicles are tracked with an extended Kalman filter method, for obtaining the moving trajectories of the vehicles; wherein in the extended Kalman filter method, a center position of the vehicle license plate and speed information are handled as system state variables;
wherein in the step S 4 , it is defined that if the vehicle salient part is continuously tracked for no less than 3 frames, then the vehicle salient part is in a stable state;
wherein searching the observed value of the vehicle has rules of:
RULE 1 : if a stable vehicle license plate is detected near the predicted position while a vehicle light state is non-stable, then the vehicle license plate is used for representing the vehicle and as the observed value thereof; meanwhile, a stable state of the vehicle light and a relative position between the vehicle light and the vehicle license plate are updated;
RULE 2 : if the vehicle license plate is not detected near the predicted position or a non-stable vehicle license plate is detected, then the vehicle light is further searched; if the vehicle light is stable, then the vehicle light is used for representing the vehicle and as the observed value thereof;
RULE 3 : if the stable vehicle license plate is detected while the vehicle searched is also stable, then distances thereof between the predicted positions are respectively calculated, and a shorter one is used as the observed value; meanwhile, the relative position between the vehicle light and the vehicle license plate is updated;
RULE 4 : if the non-stable vehicle license plate is detected near the predicted position and the vehicle light is non-stable, then a position of the vehicle license plate is used as the observed value; and
RULE 5 : if the vehicle license plate is not detected near the predicted position and the vehicle light is non-stable, then the object is regarded as being missed; if the object is continuously missed for 3 frames, then the object is regarded as being out of the road traffic scene.

3. The method, as recited in claim 2 , wherein in the step S 5 , certain detecting areas are set for different violation types; meanwhile, a plurality of evidence images and violation information are stored;
wherein a method for setting the detecting areas of different violations and analyzing the moving trajectories comprises steps of:
(1) for red light running: setting two detecting areas respectively in front of and behind a stop line, recording and analyzing the moving trajectories of each vehicle, wherein if a signal light of a current driveway is a red light, and the moving trajectories passes the two detecting areas in sequence, then the vehicle is regarded as running the red light; extracting three overall images of the vehicle in front of, at and behind the stop line, and a close-up image of the vehicle; calling a vehicle license plate recognizing program for recognizing the vehicle license plate; finally combining the images and marking the violation information comprising a violation time, an intersection location, the driveway and a vehicle license number, for post-treatment by a traffic management department;
(2) for line-pressing driving: marking driveway line areas where line pressing during driving is forbidden; recording and analyzing the moving trajectory of each vehicle, wherein if the moving trajectory passes through the driveway line area, then the vehicle is regarded as being line-pressing driving; extracting an overall image of the vehicle in a line pressing state and a close-up image of the vehicle; calling the vehicle license plate recognizing program for recognizing the vehicle license plate; finally combining the images and marking the violation information, for post-treatment by the traffic management department; and
(3) for improper driveway driving: setting two detecting areas respectively at two driveways, wherein if the moving trajectory passes through the two detecting areas in sequence, then the vehicle is regarded as being improper driveway driving; extracting three overall images of the vehicle before, during and after driveway changing, and a close-up image of the vehicle; calling the vehicle license plate recognizing program for recognizing the vehicle license plate; finally combining the images and marking the violation information, for post-treatment by the traffic management department;
thereby, traffic violation detection based on the salient parts is completed.

4. The method, as recited in claim 2 , wherein:
in the step S 1 , calibrating the road traffic scene comprises selecting a high definition video clip shot by the monitoring camera, wherein a definition thereof is 2592×1936, a video scene covers an intersection of three-driveway-roads; for obtaining physical coordinate parameters related to the images, using a built-in camera calibrating function of OpenCV (an Intel open source computer visual library) for calibrating the road traffic scene; marking image areas according to marked lines of roads with known sizes in the image, so as to inter-transform between an image coordinate system and a physical coordinate system, for obtaining physical coordinates of all positions in the image; furthermore, reading the video with a function of the OpenCV and importing into a computer;
in the step S 2 , localizing all the vehicle license plates in the video sequence comprises using gray-scale images of black vehicle license plates with white words and white vehicle license plates with black words as color gray-scale images; for blue and yellow vehicle license plates, firstly using a following formula for transforming an image space into a certain color space;
C x,y =∥B x,y −min{ R x,y , G x,y }∥

then using a Sobel operator for calculating an image gradient in a vehicle license plate colorful image; and using a sliding window for scanning a gradient image, so as to obtain an average gradient within the sliding window; wherein a size of the sliding window equals to a size of the vehicle license plate; by using a calibrating result of the OpenCV, the size of the vehicle license plate is obtained;
finally, determining an area size of the vehicle in a scoring image, using a non-maximum suppression method for obtaining a local maximum value within a vehicle area, wherein if the local maximum value is higher than a pre-determined scoring threshold, then the local maximum value is set as a center point; a vehicle license plate area is obtained based on international vehicle license plate size information, and the vehicle license p

Assignees

• Cloud Computing Center Chinese Acad Of Sciences
• Inst Automation Cas

Inventors

• Wang Feiyue
• Tian Bin
• Li Ye
• Li Bo
• Wang Kunfeng
• Xiong Gang
• Zhu Fenghua
• Hu Bin

Classifications

• G08G1/0175 Primary

  by photographing vehicles, e.g. when violating traffic rules · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06T2207/20164

  Salient point detection; Corner detection · CPC title

• G06T2207/20024

  Filtering details · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title