Systems and methods for detecting and dynamically mitigating driver fatigue

Patent metadata

Field	Value
Publication number	US-11491994-B2
Application number	US-202017024998-A
Country	US
Kind code	B2
Filing date	Sep 18, 2020
Priority date	Dec 19, 2018
Publication date	Nov 8, 2022
Grant date	Nov 8, 2022

Abstract

This technology relates to dynamically detecting, managing and mitigating driver fatigue in autonomous systems. For instance, interactions of a driver in a vehicle may be monitored to determine a distance or time when primary tasks associated with operation of the vehicle or secondary tasks issued by the vehicle computing were last performed. If primary tasks or secondary tasks are not performed within given distance thresholds or time limits, then one or more secondary tasks are initiated by the computing device of the vehicle. In another instance, potential driver fatigue, driver distraction or overreliance on an automated driving system is detected based on gaze direction or pattern of a driver. For example, a detected gaze direction or pattern may be compared to an expected gaze direction or pattern given the surrounding environment in a vicinity of the vehicle.

First claim

The invention claimed is:

1. An interactive cognitive task method for a vehicle capable of partially or fully autonomous driving, the method comprising:
determining, by one or more processors of a self-driving system of the vehicle, a primary task demand metric for a primary task associated with oversight of vehicle operation by a driver of the vehicle;
evaluating, by the one or more processors, the primary task demand metric in view of a driving context regarding one or more current driving conditions, to identify a measure of a level of driver interactivity indicating how much the driver is engaged in the primary task;
determining, by the one or more processors, when to trigger requests for the driver to perform an interactive cognitive task assigned to the driver based on the measure of the level of driver interactivity, wherein a frequency of triggering the requests is based on a time period between interactive cognitive tasks assigned to the driver or a distance travelled by the vehicle between interactive cognitive tasks assigned to the driver;
when each of the requests is triggered, determining, by the one or more processors, whether the driver correctly performed the interactive cognitive task within a predetermined time period; and
when it is determined that the driver did not correctly perform the interactive cognitive task within the predetermined time period, sending, by the one or more processors, a communication indicating that the driver should be contacted to check on the driver's alertness.

2. The method of claim 1 , wherein the one or more current driving conditions include at least one of (i) a speed of the vehicle, (ii) a time of day, or (iii) a traffic condition in an external environment within a vicinity of the vehicle.

3. The method of claim 1 , wherein the driving context includes a driver profile of the driver of the vehicle.

4. The method of claim 3 , wherein the driver profile includes at least one of age, language spoken, or experience level.

5. The method of claim 3 , wherein the driver profile indicates a likelihood of driver attentiveness according to road type.

6. The method of claim 1 , further comprising, prior to triggering each request for the driver of the vehicle, selecting the interactive cognitive task from a set of interactive cognitive tasks based on a complexity of the driving context.

7. The method of claim 1 , wherein the interactive cognitive task is assigned to the driver by determining a timing for assigning the interactive cognitive task.

8. The method of claim 7 , wherein determining the timing for assigning the interactive cognitive task includes evaluating a driving situation.

9. The method of claim 8 , wherein evaluating the driving situation includes identifying whether a complex driving scenario will occur.

10. The method of claim 8 , wherein the timing for assigning the interactive cognitive task is either before a particular driving scenario will occur or after the particular driving scenario has occurred.

11. The method of claim 8 , wherein determining the timing for assigning the interactive cognitive task includes further includes evaluating at least one of map data, road geometry or a particular object in a vicinity of the vehicle.

12. The method of claim 1 , further comprising the one or more processors initiating an additional action when it is determined that the driver did not correctly perform the interactive cognitive task within the predetermined time period.

13. The method of claim 12 , wherein the additional action includes at least one of increasing volume of a radio or music player in the vehicle, opening a driver's side window of the vehicle, activating a horn of the vehicle, vibrating a driver's seat of the vehicle or sounding a distinctive alarm.

14. The method of claim 1 , wherein the interactive cognitive task is assigned to the driver by selecting the predetermined time period in accordance with a complexity of the primary task.

15. The method of claim 1 , wherein the communication is sent to alert a customer service representative.

16. The method of claim 15 , wherein the communication is sent to alert a dispatcher.

17. The method of claim 1 , wherein the frequency of triggering the requests decreases when the measure of the level of driver interactivity increases, and the frequency of triggering the requests increases when the measure of the level of driver interactivity decreases.

18. A vehicle comprising:
a driving system including a steering subsystem, an acceleration subsystem and a deceleration subsystem to control driving of the vehicle;
a perception system including one or more sensors configured to detect objects in an environment external to the vehicle;
a positioning system configured to determine a current position of the vehicle; and
a control system including one or more processors, the control system operatively coupled to the driving system, the perception system and the positioning system, the control system being configured to:
determine a primary task demand metric for a primary task associated with oversight of vehicle operation by a driver of the vehicle;
evaluate the primary task demand metric in view of a driving context regarding one or more current driving conditions, to identify a measure of a level of driver interactivity indicating how much the driver of the vehicle is engaged in the primary task;
determine when to trigger requests for the driver to perform an interactive cognitive task assigned to the driver based on the measure of the level of driver interactivity, wherein a frequency of triggering the requests is based on a time period between interactive cognitive tasks assigned to the driver or a distance travelled by the vehicle between interactive cognitive tasks assigned to the driver;
when each of the requests is triggered, determine whether the driver correctly performed the interactive cognitive task within a predetermined time period; and
when it is determined that the driver did not correctly perform the interactive cognitive task within the predetermined time period, send a communication indicating that the driver should be contacted to check on the driver's alertness.

19. The vehicle of claim 18 , wherein the control system is further configured, prior to triggering each request for the driver of the vehicle, to select the interactive cognitive task from a set of interactive cognitive tasks based on a complexity of the driving context.

20. The vehicle of claim 18 , wherein the interactive cognitive task is assigned to the driver by determining a time for assigning the interactive cognitive task.

21. The vehicle of claim 18 , wherein the control system is further configured to initiate an additional action when it is determined that the driver did not correctly perform the interactive cognitive task within the predetermined time period.

22. The vehicle of claim 18 , wherein the interactive cognitive task is assigned to the driver by selecting the predetermined time period in accordance with a complexity of the primary task.

23. The vehicle of claim 21 , wherein the additional action includes at least one of increasing volume of a radio or music player in the vehicle, opening a driver's side window of the vehicle, activating a horn of the vehicle, vibrating a driver's seat of the vehicle or sounding a distinctive alarm.

24. The vehicle of claim 18 , wherein the one or more current driving conditions include at lea

Assignees

• Waymo Llc

Inventors

• Hutchings Keith
• Mercay Julien
• Nemec Philip
• Cavelie Hans-Olav
• Panigrahi Saswat
• Hubert Renaud-Roland
• Engstrom Johan

Classifications

• B60W2540/225

  Direction of gaze · CPC title

• B60W40/02

  related to ambient conditions · CPC title

• B60W2540/229

  Attention level, e.g. attentive to driving, reading or sleeping · CPC title

• B60W50/10

  Interpretation of driver requests or demands · CPC title

• B60W2552/05

  Type of road, e.g. motorways, local streets, paved or unpaved roads · CPC title