Cost effective and robust system and method for eye tracking and driver drowsiness identification

Patent metadata

Field	Value
Publication number	US-9483695-B2
Application number	US-201013513495-A
Country	US
Kind code	B2
Filing date	Dec 2, 2010
Priority date	Dec 2, 2009
Publication date	Nov 1, 2016
Grant date	Nov 1, 2016

Abstract

A cost-effective and robust method for localizing and tracking drowsiness state of the eyes of driver by using images captured by near infrared (IR) camera disposed on the vehicle, the said method comprising the processor implemented steps of: Real-time tracking of the face and localizing eye bounding box within the face bounding box in the captured image by comparing the gray values with threshold using the segmentation process; tracking the eyes by computing the centroid of the eye, target model histogram and target candidate model histogram for one location to another by comparing them to identify distance and calculating the displacement of the target center by the weighted means, wherein the target model histogram and target candidate model histogram are computed based on the feature space; and detecting the drowsiness state of the eyes using histogram equalization, Morphological operations and texture based parameters using histogram and grey level co-occurrence matrices.

First claim

We claim:

1. A computer implemented method for determining in real time a drowsiness state of a driver while driving by using images captured by a near infrared (IR) camera disposed on a vehicle, the method comprising:
determining a face bounding box by determining face coordinates using a segmentation process by collecting one or more features of a face and by determining a face height based on a difference between at least one of a nose tip position, an eye brow position, and co-ordinates of the eye brow position;
real time tracking of the face by collecting grey values of features of the face greater than threshold values of said features, obtained from the segmentation process;
tracking the eyes by computing a centroid of the eye, and calculating a target model histogram and a target candidate model histogram based on a range of intensity of a histogram equalized image and a morphology transformed image;
real time tracking of the eyes within a face bounding box and collecting a histogram equalization and a morphology transformation in the face bounding box;
calculating the distance between the target model histogram and the target candidate model histogram and calculating a displacement of the target centre; and
detecting a drowsiness state of a driver from the eyes by using at least one of histogram equalization, morphological operations and texture based parameters by using histogram and grey level co-occurrence matrices.

2. The method of claim 1 , wherein performing the histogram equalization and the morphological operation results in a contrast enhancement of the face bounding box.

3. The method of claim 1 , further comprising generating an alert for warning the driver based on the drowsiness state of the driver.

4. The method of claim 1 , wherein the near IR camera is disposed outside or inside the vehicle, and wherein the near IR camera faces towards the driver.

5. The method of claim 1 , wherein the tracking of the eyes in real time further comprise detecting a change in head position in a current frame with respect to a previous frame, wherein the change in the head position is detected using the Kernel tracking algorithm.

6. The method of claim 1 , wherein detecting the drowsiness state of the driver further comprises:
extending a bounding box of the eyes upwards to a centroid of the eyebrows;
applying the histogram equalization on the bounding box of the eyes;
eliminating a brighter pupil effect of a histogram equalized image by computing a line erosion of the histogram equalized image with a structuring element, wherein a width of the structuring element is equal to ⅓ of eyebrow width and a height of the structuring element is equal to one; and
using a histogram based approach to texture analysis which is based on the intensity value concentrations on all or part of an image represented as a histogram to identify the state of the eye, wherein the value of uniformity or Angular secondary moment (ASM) texture parameter occurs high for closed eye and low for open eye.

7. A system for determining drowsiness state of a driver for avoiding accidents by using images captured by a near infrared (IR) camera disposed on a vehicle, the system comprises:
a processor; and
a memory coupled to the processor, wherein the processor is capable of executing programmed instructions stored in the memory to:
determine a face bounding box by determining face coordinates using a segmentation process by collecting one or more features of a face and determine a face height based on a difference between at least one of a nose tip position, an eye brow position, and co-ordinates of the eye brow position;
track the face real time by collecting grey values of features of the face greater than threshold values of said features, obtained from the segmentation process;
track the eyes by computing a centroid of the eye, and calculating a target model histogram and a target candidate model histogram based on a range of intensity of a histogram equalized image and a morphology transformed image, wherein the eyes are tracked real time within a face bounding box and a histogram equalization and a morphology transformation in the face bounding box are collected;
calculate the distance between the target model histogram and the target candidate model histogram and calculating a displacement of the target centre; and
detect a drowsiness state of a driver from the eyes by using at least one of histogram equalization, morphological operations and texture based parameters by using histogram and grey level co-occurrence matrices.

8. The system of claim 7 , wherein the processor is further configured to generate an alert for warning the driver based on the drowsiness state of the driver.

Assignees

• S Chidanand K
• Bhowmick Brojeshwar
• Tata Consultancy Services Ltd

Inventors

• S Chidanand K
• Bhowmick Brojeshwar

Classifications

• A61B5/18

  for vehicle drivers {or machine operators} · CPC title

• G08B21/06

  indicating a condition of sleep, e.g. anti-dozing alarms · CPC title

• A61B5/163

  by tracking eye movement, gaze, or pupil change · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• A61B3/113 Primary

  for determining or recording eye movement · CPC title