Eye closure detection using structured illumination

Patent metadata

Field	Value
Publication number	US-9955903-B2
Application number	US-201414523166-A
Country	US
Kind code	B2
Filing date	Oct 24, 2014
Priority date	Mar 30, 2009
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

A monitoring system and method for monitoring and/or predicting drowsiness of a driver of a vehicle or a machinery operator are provided. A set of infrared, IR, or near infrared, NIR, light sources, is arranged such that an amount of the light emitted from the light source incidents on an eye of the driver or operator. The light that impinges on the eye of the driver or operator forms a virtual image of the signal sources on the eye including the sclera and/or cornea. An image sensor obtains consecutive images capturing the reflected light, wherein each image will contain glints from at least a subset or all of the light sources. A drowsiness index can be determined based on the extracted information of the glints of the sequence of images, wherein the drowsiness index indicates a degree of drowsiness of the driver or operator.

First claim

That which is claimed is:

1. A monitoring system for monitoring or predicting drowsiness of a driver of a vehicle or a machinery operator, comprising:
a set of infrared or near infrared light sources, each light source arranged to direct light towards an eye of a driver or operator;
the set of light sources adapted to emit light in separate time slots of a repetitive time frame, wherein the emitted light from each light source forms a virtual image of the respective light source on the cornea of the eye;
an image sensor arranged such that light reflected from the eye can be captured in an image, the image sensor comprising a filter to separate light emitted by the light sources from other light in the environment;
an image processing unit adapted to extract information of the light reflected from the eye; and
a determining unit adapted to:
determine an amplitude measure, the amplitude measure indicating an amplitude of an eye opening or eye closure, the amplitude measure being based on the extracted information of the light reflected from the eye,
determine an amplitude curve based on the amplitude measures for each of the images, and
determine a drowsiness index value based on the amplitude curve and a ratio between slow blinks and fast blinks.

2. The monitoring system according to claim 1 , wherein the set of light sources are synchronized.

3. The monitoring system according to claim 1 , wherein said determining unit is adapted to alert the driver or operator when the drowsiness index value exceeds a predetermined drowsiness index threshold.

4. The monitoring system according to claim 3 , further including an alert system, and wherein said determining unit is adapted to instruct said alert system to notify the driver or operator when the drowsiness index value has exceeded the predetermined drowsiness index threshold, with the notification being delivered by an alert signal comprising at least one of: an alert tone, an alert sound, an alert light signal, or a vibration signal.

5. The monitoring system according to claim 1 , wherein the set of light sources are configured to emit infrared or near infrared light at a specific wavelength.

6. The monitoring system according to claim 5 , wherein the specific wavelength is one of 750 nm, 850 nm and 950 nm.

7. A method for monitoring and predicting drowsiness of a driver of a vehicle or a machinery operator, comprising:
emitting light using a set of infrared or near infrared light sources, each light source being arranged to direct light towards an eye of a driver or operator, the emitted light being emitted in separate time slots of a repetitive time frame, wherein the emitted light from each light source forms a virtual image of the respective light source on the cornea of the eye;
capturing light reflected from the eye in an image using an image sensor, the image sensor comprising a filter to separate light emitted by the light sources from other light in the environment;
processing the images to extract information of the light reflected from the eye;
determining an amplitude measure, the amplitude measure indicating an amplitude of an eye opening or eye closure, the amplitude measure being based on the extracted information of the light reflected from the eye;
determining an amplitude curve based on the amplitude measures for each of the images; and
determining a drowsiness index value based on the amplitude curve and a ratio between slow blinks and fast blinks.

8. The method according to claim 7 , wherein said step of emitting light comprises synchronizing the set of light sources.

9. The method according to claim 7 , further comprising alerting the driver or operator when the drowsiness index value exceeds a predetermined drowsiness index threshold.

10. The method according to claim 9 , wherein said alerting the driver or operator includes generating an alert signal comprising at least one of: an alert tone, an alert sound, an alert light signal, or a vibration signal.

11. The method according to claim 7 , further comprising emitting light with a specific wavelength.

12. The method according to claim 11 , wherein the specific wavelength is one of 750 nm, 850 nm and 950 nm.

13. A non-transitory computer-readable storage medium storing computer-executable instructions for performing a method for monitoring or predicting drowsiness of a driver of a vehicle or a machinery operator, comprising:
emitting light using a set of infrared or near infrared light sources, each light source being arranged to direct light towards an eye of a driver or operator, the emitted light being emitted in separate time slots of a repetitive time frame, wherein the emitted light from each light source forms a virtual image of the respective light source on the cornea of the eye;
capturing light reflected from the eye in an image using an image sensor, the image sensor comprising a filter to separate light emitted by the light sources from other light in the environment;
processing the images to extract information of the light reflected from the eye;
determining an amplitude measure, the amplitude measure indicating an amplitude of an eye opening or eye closure, the amplitude measure being based on the extracted information of the light reflected from the eye;
determining an amplitude curve based on the amplitude measures for each of the images; and

determining a drowsiness index value based on the amplitude curve and a ratio between slow blinks and fast blinks.

Assignees

• Tobii Ab

Inventors

• Kobetski Miroslav
• Holmberg Johnny
• Blixt Peter
• Elvesjö John Mikael

Classifications

• G06V10/143

  Sensing or illuminating at different wavelengths · CPC title

• G08B21/06

  indicating a condition of sleep, e.g. anti-dozing alarms · CPC title

• A61B5/18 Primary

  for vehicle drivers {or machine operators} · CPC title

• Y02T10/84

  Data processing systems or methods, management, administration · CPC title

• G06K9/2018

  Physics · mapped topic