Explainability of autonomous vehicle decision making

Patent metadata

Field	Value
Publication number	US-11577746-B2
Application number	US-202016821115-A
Country	US
Kind code	B2
Filing date	Mar 17, 2020
Priority date	Jan 31, 2020
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

A processor is configured to execute instructions stored in a memory to determine, in response to identifying vehicle operational scenarios of a scene, an action for controlling the AV, where the action is from a selected decision component that determined the action based on level of certainty associated with a state factor; generate an explanation as to why the action was selected, such that the explanation includes respective descriptors of the action, the selected decision component, and the state factor; and display the explanation in a graphical view that includes a first graphical indicator of a world object of the selected decision component, a second graphical indicator describing the state factor, and a third graphical indicator describing the action.

First claim

What is claimed is:

1. An apparatus for use in traversing a vehicle transportation network by an autonomous vehicle (AV), comprising:
a memory; and
a processor, the processor configured to execute instructions stored in the memory to:
determine, in response to identifying vehicle operational scenarios of a scene, an action for controlling the AV, wherein the action is from a selected decision component that determined the action based on level of certainty associated with a state factor;
generate an explanation as to why the action was selected, wherein the explanation comprises respective descriptors of the action, the selected decision component, and the state factor;
display the explanation in a graphical view, wherein the graphical view includes a first graphical indicator of a world object of the selected decision component, a second graphical indicator describing the state factor, and a third graphical indicator describing the action, wherein the second graphical indicator is an icon displayed over an arrow indicating a trajectory of the world object of the selected decision component, and wherein the state factor is the trajectory and the icon indicates uncertainty with respect to the trajectory; and
control the AV to traverse the vehicle transportation network using the action after displaying the explanation.

2. The apparatus of claim 1 , wherein to determine, in response to identifying the vehicle operational scenarios, the action for controlling the AV comprises to:
identify distinct vehicle operational scenarios;
instantiate decision components,
wherein each of the decision components is an instance of a respective decision problem that models a respective distinct vehicle operational scenario of the distinct vehicle operational scenarios, and
wherein the each of the decision components maintains a respective state describing the respective vehicle operational scenario;

receive respective candidate vehicle control actions from the decision components; and
select the action from the respective candidate vehicle control actions,
wherein the action is from the selected decision component of the decision components, and
wherein the action is used to control the AV to traverse a portion of the vehicle transportation network.

3. The apparatus of claim 1 , wherein the instructions further comprise instructions to:
control the AV to traverse the vehicle transportation network using the action before the displaying the explanation.

4. The apparatus of claim 1 , wherein the first graphical indicator of the world object being at least one of a bounding box around the world object or a shading of the world object.

5. The apparatus of claim 1 , wherein the world object being a portion of the vehicle transportation network, and wherein the first graphical indicator being a shading of the portion of the vehicle transportation network.

6. The apparatus of claim 1 , wherein the state factor describes a predicted action of the world object and wherein the second graphical indicator depicts the state factor.

7. The apparatus of claim 1 , wherein the state factor describes a priority at an n-way stop and wherein the second graphical indicator being a numerical number indicating a priority of the world object at the n-way stop.

8. A method for use in traversing a vehicle transportation network by an autonomous vehicle (AV), comprising:
determining, in response to identifying vehicle operational scenarios of a scene, an action for controlling the AV, wherein the action is from a selected decision component that determined the action based on a level of certainty associated with a state factor;
generating an explanation as to why the action was selected, wherein the explanation comprises respective descriptors of the action, the selected decision component, and the state factor;
receiving, at an autonomous system, the explanation;
displaying the explanation to the tele-operator in a graphical map view, wherein the graphical map view includes a first graphical indicator of a world object of the selected decision component, a second graphical indicator describing the state factor, and a third graphical indicator describing the action;
outputting the explanation in a dialog window, wherein information from the AV is displayed as text in the dialog window, wherein the world object is displayed using a highlight in the graphical map view and wherein the explanation includes one or more words describing the world object and are formatted using the highlight and the world object is highlighted using a color, and wherein the one or more words are underlined using the same color; and
controlling the AV to traverse the vehicle transportation network using the action.

9. The method of claim 8 , wherein the graphical map view is displayed as an overlay of a satellite view of the scene.

10. The method of claim 9 , wherein the explanation is displayed to the tele-operator in response to the tele-operator responding to a request from the AV for assistance.

11. A method for use in traversing a vehicle transportation network by an autonomous vehicle (AV), comprising:
determining, in response to identifying vehicle operational scenarios of a scene, an action for controlling the AV, wherein the action is from a selected decision component that determined the action based on level of certainty associated with a state factor;
generating an explanation as to why the action was selected, wherein the explanation comprises respective descriptors of the action, the selected decision component, and the state factor; and
outputting the explanation in a haptic modality by vibrating, using a pattern, at least one of a seat or a steering wheel of the AV, wherein the pattern includes a first vibration pattern indicative of a world object of the selected decision component, a second vibration pattern describing the state factor, and a third vibration pattern describing the action.

12. The method of claim 11 , wherein the pattern is indicative of a combination of a certainty or uncertainty and the state factor.

13. The method of claim 11 , wherein one side of the at least one of the seat or the steering wheel of the AV is vibrated, wherein the one side corresponds to a location of the world object with respect to the AV.

Assignees

• Nissan North America Inc
• Univ Massachusetts
• Nasa
• Renault Sas

Inventors

• Wray Kyle Hollins
• Bentahar Omar
• Vagadia Astha
• Cesafsky Laura
• Jamgochian Arec
• Witwicki Stefan
• Baig Najamuddin Mirza
• Gyorfi Julius S
• Zilberstein Shlomo
• Sharma Sparsh

Classifications

• B60W2050/146

  Display means · CPC title

• B60W30/18159

  Traversing an intersection · CPC title

• B60W50/16 Primary

  Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal · CPC title

• B60W60/0011

  involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles · CPC title

• B60W30/18154

  Approaching an intersection · CPC title