System and method for providing adaptive trust calibration in driving automation

Patent metadata

Field	Value
Publication number	US-11498591-B2
Application number	US-202016797337-A
Country	US
Kind code	B2
Filing date	Feb 21, 2020
Priority date	Sep 13, 2019
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

A system and method for providing adaptive trust calibration in driving automation that include receiving image data of a vehicle and vehicle automation data associated with automated of driving of the vehicle. The system and method also include analyzing the image data and vehicle automation data and determining an eye gaze direction of a driver of the vehicle and a driver reliance upon automation of the vehicle and processing a Markov decision process model based on the eye gaze direction and the driver reliance to model effects of human trust and workload on observable variables to determine a control policy to provide an optimal level of automation transparency. The system and method further include controlling autonomous transparency of at least one driving function of the vehicle based on the control policy.

First claim

The invention claimed is:

1. A computer-implemented method for providing adaptive trust calibration in driving automation comprising:
electronically receiving image data of a vehicle and vehicle automation data associated with automated of driving of the vehicle;
electronically analyzing the image data and vehicle automation data and determining an eye gaze direction of a driver of the vehicle and a driver reliance as a value that indicates a driver's reliance upon an autonomous operation of at least one driving function of the vehicle or that indicates non reliance by the driver upon the autonomous operation of at least one driving function and manual takeover of the vehicle;
electronically processing a Markov decision process model based on the eye gaze direction and the driver reliance to model effects of workload, wherein the workload is a level of cognitive load that is required by the driver to process information associated with the autonomous operation of the at least one driving function of the vehicle to determine a control policy to provide a particular level of automation transparency; and
electronically controlling the autonomous transparency of at least one driving function of the vehicle based on the control policy, wherein a human machine interface is provided with at least one augmented reality annotation cue that is presented to the driver through at least one display device of the vehicle to adjust the autonomous transparency with respect to the autonomous operation of the at least one driving function.

2. The computer-implemented method of claim 1 , wherein electronically receiving image data of the vehicle includes receiving image data associated with an interior cabin of the vehicle that includes captured images of eyes of the driver, wherein image data is also associated with an external surrounding environment of the vehicle.

3. The computer-implemented method of claim 2 , wherein data associated with the captured images of the eyes of the driver are electronically analyzed using a linear model that takes into account an evaluation of specific areas of the eyes of the driver of the vehicle as the vehicle is being operated.

4. The computer-implemented method of claim 2 , further including electronically receiving LiDAR data associated with the external surrounding environment of the vehicle, wherein the LiDAR data is aggregated with the image data that is associated with the external surrounding environment of the vehicle to determine a driving scene of the vehicle, wherein a driving scene complexity is determined and associated with the driving scene of the vehicle based on complexity values that are associated with at least one of: roadways, traffic conditions, object complexities, and intersection conditions.

5. The computer-implemented method of claim 4 , wherein the value associated with the driver reliance is based on whether the driver completely relies on automation or does not completely rely on automation, wherein the value associated with the driver reliance is determined based on types of semi-autonomous and manual operations completed by the vehicle or the driver.

6. The computer-implemented method of claim 5 , wherein the eye gaze direction of the driver, the automation transparency, and the driving scene complexity are utilized to model the effects of the workload.

7. The computer-implemented method of claim 6 , further including electronically predicting a future workload based on the workload at a current point in time.

8. The computer-implemented method of claim 7 , wherein electronically processing the Markov decision process model includes electronically executing a machine learning probabilistic framework using the Markov decision process model as a partially observable model through hidden states, wherein the dynamics of human workload at the current point in time and the future workload are modeled using the Markov decision process model.

9. The computer-implemented method of claim 8 , wherein electronically processing the Markov decision process model includes electronically processing the Markov decision process model as a multiple tuple model that includes a finite set of states of the driver, the finite set of actions, and a set of observations, wherein the Markov decision process model includes a reward function and a discount factor that are used to process the control policy based on the effects of workload on observable variables.

10. A system for providing adaptive trust calibration in driving automation comprising:
a memory storing instructions when executed by a processor cause the processor to:
electronically receive image data of a vehicle and vehicle automation data associated with automated of driving of the vehicle;
electronically analyze the image data and vehicle automation data and determining an eye gaze direction of a driver of the vehicle and a driver reliance as a value that indicates a driver's reliance upon an autonomous operation of at least one driving function of the vehicle or that indicates non reliance by the driver upon the autonomous operation of at least one driving function and manual takeover of the vehicle;
electronically process a Markov decision process model based on the eye gaze direction and the driver reliance to model effects of workload, wherein the workload is a level of cognitive load that is required by the driver to process information associated with the autonomous operation of the at least one driving function of the vehicle to determine a control policy to provide an optimal level of automation transparency; and
electronically control the autonomous transparency of at least one driving function of the vehicle based on the control policy, wherein a human machine interface is provided with at least one augmented reality annotation cue that is presented to the driver through at least one display device of the vehicle to adjust the autonomous transparency with respect to the autonomous operation of the at least one driving function.

11. The system of claim 10 , wherein electronically receiving image data of the vehicle includes receiving image data associated with an interior cabin of the vehicle that includes captured images of eyes of the driver, wherein image data is also associated with an external surrounding environment of the vehicle.

12. The system of claim 11 , wherein data associated with the captured images of the eyes of the driver are electronically analyzed using a linear model that takes into account an evaluation of specific areas of the eyes of the driver of the vehicle as the vehicle is being operated.

13. The system of claim 11 , further including electronically receiving LiDAR data associated with the external surrounding environment of the vehicle, wherein the LiDAR data is aggregated with the image data that is associated with the external surrounding environment of the vehicle to determine a driving scene of the vehicle, wherein a driving scene complexity is determined and associated with the driving scene of the vehicle based on complexity values that are associated with at least one of: roadways, traffic conditions, object complexities, and intersection conditions.

14. The system of claim 13 , wherein the value associated with the driver reliance is based on whether the driver completely relies on automation or does not completely rely on automation, wherein the value associated with the driver reliance is determined based on types of semi-autonomous and manual operations completed by the vehicle or the driver.

15. The system of claim 14 , wherein the eye gaze direction of the driver, the automation transparenc

Assignees

• Honda Motor Co Ltd

Inventors

• Misu Teruhisa

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W2540/225

  Direction of gaze · CPC title

• G06V40/19

  Sensors therefor · CPC title

• B60R2011/0003

  inside the vehicle · CPC title