Explainability of autonomous vehicle decision making

Patent metadata

Field	Value
Publication number	US-11714971-B2
Application number	US-202016778890-A
Country	US
Kind code	B2
Filing date	Jan 31, 2020
Priority date	Jan 31, 2020
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

A processor is configured to execute instructions stored in a memory to identify distinct vehicle operational scenarios; instantiate decision components, where each of the decision components is an instance of a respective decision problem, and where the each of the decision components maintains a respective state describing the respective vehicle operational scenario; receive respective candidate vehicle control actions from the decision components; select an action from the respective candidate vehicle control actions, where the action is from a selected decision component of the decision components, and where the action is used to control the AV to traverse a portion of the vehicle transportation network; and generate an explanation as to why the action was selected, where the explanation includes respective descriptors of the action, the selected decision component, and a state factor of the respective state of the selected decision component.

First claim

What is claimed is:

1. An apparatus for traversing a vehicle transportation network by an autonomous vehicle (AV), comprising:
a memory; and
a processor, the processor configured to execute instructions stored in the memory to:
identify distinct vehicle operational scenarios representing vehicle operational scenarios corresponding to environment or external objects;
instantiate decision components,
wherein each of the decision components is an instance of a respective decision problem that models the respective distinct vehicle operational scenario of the distinct vehicle operational scenarios,
wherein each of the decision components comprises a respective algorithm configured to generate an output representing a respective candidate vehicle control action based on the respective distinct vehicle operational scenario,
wherein the each of the decision components stores respective state information representing a configuration of the respective vehicle operational scenario,
wherein at least one of the respective state information comprises state factors and corresponding values such that respective semantic meanings are associated with the state factors and the corresponding values, the state factor representing an aspect of the respective vehicle operational scenario,
wherein the state factors associated with one of the decision components include an AV position factor describing a position of the AV with respect to an intersection, an AV wait time factor describing how long the AV has been stopped at the intersection, an other-vehicle position factor describing a position of another vehicle with respect to the intersection, and an other-vehicle wait time factor describing how long the another vehicle has been stopped at the intersection, and
wherein a corresponding value for the AV position factor is selected from a set comprising “at”, “edged”, “inside”, and “goal”, a corresponding value for the AV wait time factor is selected from a set comprising “short” and “long”, a corresponding value for the other-vehicle position factor is selected from a set comprising “approaching”, “at”, “edged”, and “inside”, and a corresponding value for the other position-vehicle factor is selected from a set comprising “short” and “long”;

receive respective candidate vehicle control actions from the decision components;
select an action from the respective candidate vehicle control actions,
wherein the action is from a selected decision component of the decision components, and
wherein the action is used to control the AV to traverse a portion of the vehicle transportation network;

control the AV to traverse the portion of the vehicle transportation network using the action; and
generate an explanation as to why the action was selected,
wherein the explanation comprises respective descriptors of the action, the selected decision component, and the state factor of the respective state information of the selected decision component.

2. The apparatus of claim 1 , wherein the explanation indicates a level of certainty or uncertainty that the selected decision component associates with the state factor.

3. The apparatus of claim 1 , wherein the selected decision component maintains a prioritized list of state factors and the state factor of the explanation being either a lowest or highest priority state factor of the prioritized list of state factors.

4. The apparatus of claim 1 , wherein the selected decision component is a partially observable Markov decision process (POMDP).

5. The apparatus of claim 1 , wherein the instructions further comprise instructions to:
construct the explanation by inserting values into a template.

6. The apparatus of claim 5 , wherein the template has a format “I <action taken> because I had <importance measure> about <state factor> for <decision component>,” and
wherein each of <action taken>, <importance measure>, <state factor>, and <decision component> is a placeholder for a respective semantic descriptor.

7. The apparatus of claim 1 , wherein the instructions further comprise instructions to:
receive the action from a second selected decision component, wherein the explanation further comprises an indicator of the second selected decision component and an indicator of a state factor of the second selected decision component.

8. The apparatus of claim 1 , wherein the explanation is provided to at least one of a tele-operator of the AV or an occupant of the AV.

9. The apparatus of claim 1 , wherein the instructions further comprise instructions to:
output the explanation in at least one of a visual, a textual, or an audio format.

10. The apparatus of claim 9 , wherein the instructions further comprise instructions to:
receive a request from an occupant of the AV to output the explanation in the at least one of a visual, a textual, or an audio format.

11. The apparatus of claim 1 , wherein the instructions further comprise instructions to:
output the explanation to a log, wherein the log comprises historical actions of controlling the AV.

12. A method for use in traversing a vehicle transportation network by an autonomous vehicle (AV), the method comprising:
identifying distinct vehicle operational scenarios based on observed environment or external objects;
generating, using respective decision components, respective candidate actions; based on each of the distinct vehicle operational scenarios,
wherein the respective decision components comprise respective algorithms configured to generate the respective candidate actions based on the distinct vehicle operational scenarios,
wherein the decision components store respective state information representing a configuration of the respective vehicle operational scenario,
wherein the respective state information comprises state factors and corresponding values such that at least one semantic information is associated with the state factor and the corresponding value, the state factor representing an aspect of the respective vehicle operational scenario,
wherein the state factors associated with one of the respective decision components include at least a blocking factor describing whether a trajectory of the AV and a trajectory of another vehicle intersect at a road intersection, and a priority factor describing which of the AV or the other vehicle has a right of way at the road intersection, and
wherein a corresponding value for the blocking factor is selected from a set comprising at least “Yes” and “No”, and a corresponding value for the priority factor is selected from a set comprising “AV” and “other vehicle”;

controlling the AV to traverse a portion of the vehicle transportation network based on a selected action of the candidate actions, wherein the selected action is from a selected decision component of the respective decision components; and
generating an explanation of the selected action based on the semantic information.

13. The method of claim 12 , wherein generating, using the respective decision components, candidate actions comprises:
identifying a decision problem for one of the distinct vehicle operational scenarios, wherein the decision problem provides a policy usable by the one of the distinct vehicle operational scenarios,
wherein the policy provides an action for controlling the AV given a currently observed state or a predicted state; and

instantiating the respective decision problem to generate one of the respective decision components, wherein the one of the respective decision components provides the respective candidate action based on the policy.

14. The method of claim 12 , wherein the explanation inc

Assignees

• Nissan North America Inc
• Univ Massachusetts
• Renault Sas

Inventors

• Wray Kyle Hollins
• Witwicki Stefan
• Zilberstein Shlomo
• Bentahar Omar
• Jamgochian Arec

Classifications

• G05D1/0221 Primary

  involving a learning process · CPC title

• G05D1/0219

  ensuring the processing of the whole working surface · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W60/0011

  involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles · CPC title

• G06F40/186

  Templates · CPC title