Method and device for detecting safe driving state of driver

Patent metadata

Field	Value
Publication number	US-9963153-B2
Application number	US-201515112166-A
Country	US
Kind code	B2
Filing date	Jan 15, 2015
Priority date	Jan 15, 2014
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

Disclosure is a method for detecting the safety driving state of a driver, the method comprises the following steps: (a) detecting the current sight direction of a driver in real time and acquiring a scene image signal in a front view field of the driver when a vehicle runs; (b) processing the acquired current road scene image signal according to a visual attention calculation model to obtain the expected attention distribution of the driver under the current road scene; and (c) performing fusion analysis on the real-time detected current sight direction of the driver in the step (a) and the calculated expected attention distribution of the driver in step (b), and judging whether the current driver is in a normal driving state and whether the driver can timely make a proper response to the sudden road traffic accident. The device is used for implementing the method and has the advantages of simple principle, easy realization, direct reflection of the real driving state of a driver, and improvement of the driving safety.

First claim

What is claimed is:

1. A method for detecting a safety driving state of a driver, comprising steps of:
(a) detecting a current sight direction of a driver in real time and acquiring a scene image signal in a front view field of the driver when a vehicle runs;
(b) processing an acquired current road scene image signal according to a visual attention calculation model to obtain an expected attention distribution of the driver under a current road scene; and
(c) performing fusion analysis on a real-time detected current sight direction of the driver in the step (a) and a calculated expected attention distribution of the driver in step (b), and judging whether the driver is in a normal driving state and whether the driver can timely make a proper response to a sudden road traffic accident.

2. The method as recited in claim 1 , wherein whether the driver timely makes a response to an important traffic event in a current scene or not is evaluated in a quantitative mode; and a quantitative evaluation is specifically to analyze whether a sight of the driver falls into an area of an important traffic event in a scene or not and analyze a fall proportion and speed at any certain moment.

3. The method as recited in claim 1 , wherein the current sight direction of the driver is detected in real time by a sight capturing device, and the scene image signal in the front view field of the driver when the vehicle runs is acquired in real time by a vehicle-mounted forward-looking camera.

4. The method as recited in claim 1 , wherein a driver attention calculation method based on significant traffic events and significance is adopted in the step (b) and comprises the following specific steps:
(f.1) starting image acquisition, and storing a current m×n forward-looking image;
(f.2) generating three dynamic arrays P 1 , P 2 and P 3 and three m×n matrixes for storing intermediate data: Map 1 , Map 2 and Map 3 , wherein the forward-looking image roughly comprises three types of driver attention areas: attention areas R 0 related to traffic events, significant areas R S in a forward-looking image and fixed areas R OF often noticed by the driver, and the matrixes Map 1 , Map 2 and Map 3 are used for storing the three types of areas;
(f.3) determining R OF type attention areas {R OF i } and generating a first attention distribution map Map 1 , wherein Map 1 ( x )=1, if xϵ{R OF i }; else Map 1 ( x )=0;
(f.4) storing indexes of the areas in Map 1 into the array P 1 in sequence, wherein P 1 ={R OF 1 , R OF 2 , . . . , R OF N };
(f.5) generating traffic event related attention areas: detecting and tracking R 0 type areas {R O k i } of front lanes, vehicles, pedestrians, traffic signs and the like by using a computer vision method, wherein k=1, 2, 3 or 4, representing four types of areas of front lanes, vehicles, pedestrians and traffic signs; and generating a second attention distribution map: Map 2 ( x )=1, if xϵ{R O k i }; else Map 2 ( x )=0;
(f.6) storing the indexes of the areas in Map 2 into the array P 2 in sequence, wherein P 2 ={R O 1 1 , R O 1 2 , . . . , R O 1 M ; R O 2 1 , R O 2 2 , . . . , R O 2 N ; . . . ; R O 4 1 , R O 4 2 , . . . , R O 4 Q };
(f.7) calculating a significant area of a forward-looking image 1 ( t ) according to a visual significance algorithm, and generating a significant map Map 3 of binary areas {R S i }: Map 3 ( x )=1, if xϵ{R S i }; else Map 3 ( x )=0; and
(f.8) storing the indexes of the areas in Map 3 into the array P 3 in sequence, wherein the array P 3 ={R S 1 , R s 2 , . . . , R s N }.

5. The method as recited in claim 4 , wherein in the step (c), an interactive processes between traffic related areas and a driver attention distribution and between the significant areas and the driver attention distribution are uniformly modeled, and a specific flow includes:
“needing to be noticed” degrees of areas being modeled as activities of areas by means of a neuron activity description method in physiology, expressed by V, wherein when the driver does not notice these areas, activities V of areas rises according to a certain law, wherein when the driver puts the sight to a certain area, the activity of the area is quickly pulled down, wherein first type of areas need to be noticed by the driver, and if these areas are not noticed for long time, the activities V rise a lot and an alarm is given when the activities V exceed a certain threshold, wherein second type of areas are not expected to be noticed by the driver for a long term, and if the driver notices these areas for a long term, the activities V continuously decline and an alarm is given when the activities V are lower than a certain threshold;
for the first type of areas, activities thereof being modeled into a dynamic process below according to an evolution law of time:

∂
V

∂
t

=

-

[

α
′

+
ϕγ

]

⁢

(

V
-
0

)

+

β
⁡

(

1
+
μ
-
V

)

(
i
)

for the second type of areas, activities thereof being modeled as follows according to the evolution law of time:

∂
V

∂
t

=

-

[

α
′

+
ϕγ

]

Assignees

• National Univ Of Defense Technology

Inventors

• An Xiangjing
• Li Jian
• Wu Tao
• Ye Lei
• Sun Zhenping
• He Hangen

Classifications

• B60W2040/0872

  Driver physiology · CPC title

• B60W40/09 Primary

  Driving style or behaviour · CPC title

• G08B21/06

  indicating a condition of sleep, e.g. anti-dozing alarms · CPC title

• G06F17/11

  for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title

• B60W30/08

  Active safety systems} predicting or avoiding probable or impending collision {or attempting to minimise its consequences · CPC title