Predictive driver alertness assessment

Patent metadata

Field	Value
Publication number	US-11760318-B2
Application number	US-202117199052-A
Country	US
Kind code	B2
Filing date	Mar 11, 2021
Priority date	Mar 11, 2021
Publication date	Sep 19, 2023
Grant date	Sep 19, 2023

Abstract

In an exemplary embodiment, a system is provided that includes one or more first sensors, one or more second sensors, and a processor disposed onboard a vehicle. The first sensors are configured to at least facilitate obtaining first sensor data with regard to an external environment outside the vehicle. The second sensors are configured to at least facilitate obtaining second sensor data with regard to one or more eyes of a driver of the vehicle. The processor is configured to at least facilitate: determining a predicted gaze angle of the one or more eyes of the driver based on the external environment outside the vehicle, using the first sensor data; determining a measured gaze angle of the one or more eyes of the driver, using the second sensor data, and controlling one or more vehicle actions based on a comparison of the predicted and measured gaze angles.

First claim

What is claimed is:

1. A system comprising:
one or more first sensors disposed onboard a vehicle and configured to at least facilitate obtaining first sensor data with regard to an external environment outside the vehicle;
one or more second sensors disposed onboard the vehicle and configured to at least facilitate obtaining second sensor data with regard to one or more eyes of a driver of the vehicle; and
a processor disposed onboard the vehicle and configured to at least facilitate:
determining a predicted gaze angle of the one or more eyes of the driver based on the external environment outside the vehicle, using the first sensor data, wherein the predicted gaze is based on a predicted Euler gaze for pupils of the one or more eyes based on trajectory curvatures using a curvature of a road on which the vehicle is travelling;
determining a measured gaze angle of the one or more eyes of the driver, using the second sensor data, and
controlling one or more vehicle actions based on a comparison of the predicted gaze angle and the measured gaze angle.

2. The system of claim 1 , wherein:
the one or more first sensors comprise one or more exterior cameras that face outside the vehicle and that are configured to obtain external camera images of the external environment outside the vehicle; and
the one or more second sensors comprise one or more interior cameras that face inside the vehicle and that are configured to obtain internal camera images of the driver inside the vehicle.

3. The system of claim 1 , wherein the processor is further configured to at least facilitate:
determining a trajectory curvature of a roadway on which the vehicle is travelling, based on the first sensor data; and
determining the predicted gaze angle of the one or more eyes of the driver based on the trajectory curvature of the roadway.

4. The system of claim 1 , wherein the processor is further configured to at least facilitate:
determining one or more infrastructure objects of a roadway on which the vehicle is travelling, based on the first sensor data; and
determining the predicted gaze angle of the one or more eyes of the driver based on the one or more infrastructure objects.

5. The system of claim 1 , wherein the processor is further configured to at least facilitate controlling the one or more vehicle actions based on an absolute error between the predicted gaze angle and the measured gaze angle.

6. The system of claim 5 , wherein the processor is further configured to at least facilitate providing a notification when the absolute error is greater than a predetermined threshold.

7. The system of claim 5 , wherein the processor is further configured to at least facilitate automatically controlling movement of the vehicle when the absolute error is greater than a predetermined threshold.

8. The system of claim 5 , wherein the processor is further configured to at least facilitate applying automatic braking for the vehicle when the absolute error is greater than a predetermined threshold.

9. The system of claim 5 , wherein the processor is further configured to at least facilitate applying automatic steering for the vehicle when the absolute error is greater than a predetermined threshold.

10. The system of claim 5 , wherein the processor is further configured to at least facilitate adjusting a threshold for one or more automatic control features for the vehicle when the absolute error is greater than a predetermined threshold.

11. A method comprising:
obtaining first sensor data with regard to an external environment outside a vehicle;
obtaining second sensor data with regard to one or more eyes of a driver of the vehicle;
determining, via a processor, a predicted gaze angle of the one or more eyes of the driver based on the external environment outside the vehicle, using the first sensor data, wherein the predicted gaze is based on a predicted Euler gaze for pupils of the one or more eyes based on trajectory curvatures using a curvature of a road on which the vehicle is travelling;
determining, via the processor, a measured gaze angle of the one or more eyes of the driver, using the second sensor data, and
controlling, via the processor, one or more vehicle actions based on a comparison of the predicted gaze angle and the measured gaze angle.

12. The method of claim 11 , wherein:
the step of obtaining the first sensor data comprises obtaining the first sensor data via an external camera of the vehicle that faces outside the vehicle and that is configured to obtain external camera images of the external environment outside the vehicle; and
the step of obtaining the second sensor data comprises obtaining the second sensor data via an internal camera of the vehicle that faces inside the vehicle and that is configured to obtain internal camera images of the driver inside the vehicle.

13. The method of claim 11 , further comprising:
determining, via the processor, a trajectory curvature of a roadway on which the vehicle is travelling, based on the first sensor data;
wherein the step of determining the predicted gaze angle comprises determining, via the processor, the predicted gaze angle of the one or more eyes of the driver based on the trajectory curvature of the roadway.

14. The method of claim 11 , further comprising:
determining, via the processor, one or more infrastructure objects of a roadway on which the vehicle is travelling, based on the first sensor data;
wherein the step of determining the predicted gaze angle comprises determining, via the processor, the predicted gaze angle of the one or more eyes of the driver based on the one or more infrastructure objects.

15. The method of claim 11 , wherein the step of controlling the one or more vehicle actions comprises:
controlling, via the processor, the one or more vehicle actions based on an absolute error between the predicted gaze angle and the measured gaze angle.

16. The method of claim 15 , wherein the step of controlling the one or more vehicle actions comprises applying automatic braking for the vehicle, via instructions provided by the processor, when the absolute error is greater than a predetermined threshold.

17. The method of claim 15 , wherein the step of controlling the one or more vehicle actions comprises applying automatic steering for the vehicle, via instructions provided by the processor, when the absolute error is greater than a predetermined threshold.

18. A vehicle comprising:
a body;
a propulsion system configured to generate movement of the body;
one or more first sensors disposed onboard the vehicle and configured to at least facilitate obtaining first sensor data with regard to an external environment outside the vehicle;
one or more second sensors disposed onboard the vehicle and configured to at least facilitate obtaining second sensor data with regard to one or more eyes of a driver of the vehicle; and
a processor onboard the vehicle and configured to at least facilitate:
determining a predicted gaze angle of the one or more eyes of the driver based on the external environment outside the vehicle, using the first sensor data, wherein the predicted gaze is based on a predicted Euler gaze for pupils of the one or more eyes based on trajectory curvatures using a curvature of a road on which the vehicle is travelling;
determining a measured gaze angle of the one or more eyes of the driver, using the second sensor data, and
controlling one or more vehicle actions based on a comparison of the predicted gaze angle and the measured gaze angle.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Song Xiaofeng F
• Didonato Benjamin D

Classifications

• B60W40/08 Primary

  related to drivers or passengers · CPC title

• B62D15/0285 Primary

  Parking performed automatically · CPC title

• B60T7/12 Primary

  for automatic initiation; for initiation not subject to will of driver or passenger {(limiting speed of vehicles other than rail vehicles B60K31/00)} · CPC title

• B60W2540/225

  Direction of gaze · CPC title

• B60T7/14

  operated upon collapse of driver (deadman's devices for electrically propelled vehicles B60L3/02) · CPC title