Detecting visual inattention based on eye convergence

Patent metadata

Field	Value
Publication number	US-9298994-B2
Application number	US-201414151670-A
Country	US
Kind code	B2
Filing date	Jan 9, 2014
Priority date	Jan 9, 2014
Publication date	Mar 29, 2016
Grant date	Mar 29, 2016

Abstract

One embodiment of the present invention sets forth a technique for detecting when a user is not being attentive based on eye convergence. The technique includes determining a first distance from the user to a first object. The first distance may be determined using a depth sensor. The technique further includes determining a first vergence angle associated with a left eye of the user and determining a second vergence angle associated with a right eye of the user. The technique further includes determining a first eye convergence distance based on the first vergence angle, the second vergence angle, and an interocular distance between the left eye and the right eye. The technique further includes comparing the first distance to the first eye convergence distance to generate a first result. An inattention alert may be generated based on the first result.

First claim

What is claimed is:

1. A computer-implemented method for detecting when a user is not being attentive based on eye convergence, the method comprising:
determining, via a depth sensor, a first distance between a location associated with a vehicle and a first object located outside of the vehicle;
determining a second distance from the user to the first object based on the first distance;
determining a first vergence angle associated with a left eye of the user;
determining a second vergence angle associated with a right eye of the user;
determining a first eye convergence distance based on the first vergence angle, the second vergence angle, and an interocular distance between the left eye and the right eye; and
comparing the second distance to the first eye convergence distance to generate a first result.

2. The method of claim 1 , wherein comparing the second distance to the first eye convergence distance comprises determining whether the second distance is substantially similar to the first eye convergence distance.

3. The method of claim 2 , wherein the second distance is not substantially similar to the first eye convergence distance, and further comprising generating an inattention alert for the user based on the first result.

4. The method of claim 1 , wherein determining the first distance comprises acquiring a depth measurement to the first object, and the depth sensor comprises at least one of a time-of-flight depth sensor and an optical depth sensor.

5. The method of claim 1 , wherein determining the first distance comprises accessing at least one of global positioning system (GPS) data, a vehicle-to-vehicle communication protocol, a peer-to-peer communication protocol, and a geographical information database to read a distance value.

6. The method of claim 1 , further comprising:
determining an eye gaze direction of the user;
determining a third distance from the user to a second object associated with the eye gaze direction;
determining a third vergence angle associated with the left eye of the user based on the eye gaze direction;
determining a fourth vergence angle associated with the right eye of the user based on the eye gaze direction;
determining a second eye convergence distance based on the third vergence angle, the fourth vergence angle, and the interocular distance; and
comparing the third distance to the second eye convergence distance to generate a second result.

7. The method of claim 6 , further comprising, prior to determining the third distance, positioning the depth sensor based on at least one of the eye gaze direction and a user head direction, and wherein determining the first distance to the first object comprises acquiring a depth measurement to the first object with the depth sensor.

8. The method of claim 1 , wherein determining the first vergence angle comprises tracking a left pupil of the user, and determining the second vergence angle comprises tracking a right pupil of the user.

9. The method of claim 1 , wherein determining the first vergence angle comprises determining a first three-dimensional vector associated with a left pupil of the user, and determining the second vergence angle comprises determining a second three-dimensional vector associated with a right pupil of the user.

10. A system for tracking user attentiveness, comprising:
a depth sensor configured to determine a first distance between a location associated with a vehicle and a first object located outside of the vehicle;
an image sensor configured to acquire one or more images of a user; and
a processor configured to:
determine a second distance from the user to the first object based on the first distance;
determine a first vergence angle associated with a left eye of the user based on the one or more images;
determine a second vergence angle associated with a right eye of the user based on the one or more images;
determine a first eye convergence distance based on the first vergence angle, the second vergence angle, and an interocular distance between the left eye and the right eye; and
determine whether the second distance is substantially similar to the first eye convergence distance to generate a first result.

11. The system of claim 10 , wherein the second distance is not substantially similar to the first eye convergence distance, and the processor is further configured to cause an inattention alert to be generated for the user based on the first result.

12. The system of claim 11 , further comprising an alert device configured to generate the inattention alert, wherein the alert device comprises at least one of a speaker, a light-emitting device, and a vibration motor.

13. The system of claim 10 , wherein the depth sensor is configured to determine the first distance by performing at least one of a time-of-flight depth measurement and an optical depth measurement.

14. The system of claim 10 , wherein the depth sensor is configured to determine the first distance by accessing at least one of global positioning system (GPS) data, a vehicle-to-vehicle communication protocol, a peer-to-peer communication protocol, and a geographical information database to read a distance value.

15. The system of claim 10 , wherein:
the depth sensor is further configured to determine a third distance between the location associated with the vehicle and a second object associated with an eye gaze direction of the user and located outside of the vehicle; and
the processor is further configured to:
determine the eye gaze direction of the user based on the one or more images;
determine a fourth distance from the user to the second object based on the third distance;
determine a third vergence angle associated with the left eye of the user based on the one or more images;
determine a fourth vergence angle associated with the right eye of the user based on the one or more images;
determine a second eye convergence distance based on the third vergence angle, the fourth vergence angle, and the interocular distance; and
compare the fourth distance to the second eye convergence distance to generate a second result.

16. The system of claim 15 , wherein the depth sensor is further configured to, prior to determining the third distance, reposition based on at least one of the eye gaze direction and a user head direction.

17. The system of claim 16 , wherein the depth sensor is configured to determine the third distance by performing at least one of a time-of-flight depth measurement and an optical depth measurement.

18. The system of claim 10 , wherein the processor is configured to determine the first vergence angle by analyzing a left pupil of the user based on the one or more images, and determine the second vergence angle by analyzing a right pupil of the user based on the one or more images.

19. The system of claim 18 , wherein the processor is further configured to determine the first vergence angle by determining a first three-dimensional vector associated with a left pupil of the user, and determine the second vergence angle by determining a second three-dimensional vector associated with a right pupil of the user.

20. A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to detect when a user is not being attentive based on eye convergence, by performing the steps of:
determining, via a depth sensor, a first distance between a location associated with a vehicle and a first object located outside of the vehicle;
determ

Assignees

• Harman Int Ind

Inventors

• Marti Stefan
• Di Censo Davide

Classifications

• B60W2540/221

  Physiology, e.g. weight, heartbeat, health or special needs · CPC title

• A61B5/002

  Monitoring the patient using a local or closed circuit, e.g. in a room or building (A61B5/0017 takes precedence) · CPC title

• B60W40/08

  related to drivers or passengers · CPC title

• A61B5/746

  Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms · CPC title

• B60K28/066

  actuating a signalling device (B60K28/063 takes precedence) · CPC title