Learning safety and human-centered constraints in autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11613269-B2
Application number	US-201916724635-A
Country	US
Kind code	B2
Filing date	Dec 23, 2019
Priority date	Dec 23, 2019
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

Traversing a vehicle transportation network includes operating a scenario-specific operational control evaluation module instance. The scenario-specific operational control evaluation module instance includes an instance of a scenario-specific operational control evaluation model of a distinct vehicle operational scenario. Operating the scenario-specific operational control evaluation module instance includes identifying a multi-objective policy for the scenario-specific operational control evaluation model. The multi-objective policy may include a relationship between at least two objectives. Traversing the vehicle transportation network includes receiving a candidate vehicle control action associated with each of the at least two objectives. Traversing the vehicle transportation network includes selecting a vehicle control action based on a buffer value. Traversing the vehicle transportation network includes performing the selected vehicle control action, determining a preference indicator for each objective, and updating the multi-objective policy.

First claim

What is claimed is:

1. A method for use in a vehicle transportation network, the method comprising:
identifying, by an autonomous vehicle, a multi-objective policy for an intersection scenario-specific operational control evaluation model, wherein the multi-objective policy includes at least a first objective associated with a passenger comfort preference that is learned based on user override data, a second objective associated with a passenger risk preference that is learned based on the user override data, and a topological graph that shows a relationship between the first objective and the second objective, wherein an edge of the topological graph defines a constraint that restricts policies of the first objective and the second objective;
receiving, from the multi-objective policy, a candidate vehicle control action associated with the first objective and a candidate vehicle control action associated with the second objective;
determining a buffer value based on the relationship between the first objective and the second objective, wherein the buffer value includes an amount of variation of risk between the first objective and the second objective;
selecting a vehicle control action based on the buffer value, wherein the selected vehicle control action is the candidate vehicle control action associated with the first objective or the candidate vehicle control action associated with the second objective;
performing the selected vehicle control action to traverse a portion of the vehicle transportation network;
determining a preference indicator for each objective based on the performed vehicle control action;
determining a competence measure for each of the first objective and the second objective based on the performed vehicle control action to restrict a number of available vehicle control actions, wherein each of the competence measures of the first objective and the second objective is a learned probability of success; and
update the multi-objective policy based on the preference indicator and each of the competence measures of the first objective and the second objective.

2. The method of claim 1 , wherein the preference indicator is a reward value.

3. The method of claim 2 , wherein the reward value is a positive integer on a condition that the performed vehicle control action is successful.

4. The method of claim 2 , wherein the reward value is a negative integer on a condition that the performed vehicle control action is overridden.

5. The method of claim 1 , wherein the preference indicator is based on a user input.

6. The method of claim 1 , wherein the preference indicator is a change in the buffer value.

7. The method of claim 1 , wherein the preference indicator is an ordering of a plurality of objectives.

8. The method of claim 1 , wherein at least the first candidate vehicle control action or the second candidate vehicle control action is based on aggregated data.

9. The method of claim 8 , wherein the aggregated data is based on a geographic region, a user, a vehicle, a company, or a fleet of vehicles.

10. An autonomous vehicle comprising:
a processor configured to execute instructions stored on a non-transitory computer readable medium to:
identify a multi-objective policy for an intersection scenario-specific operational control evaluation model, wherein the multi-objective policy includes at least a first objective that prioritizes vehicle control actions based on a passenger comfort preference that is learned based on user override data, a second objective that prioritizes vehicle control actions based on an expected time of completion, and a topological graph that shows a relationship between the first objective and the second objective, wherein an edge of the topological graph defines a constraint that restricts policies of the first objective and the second objective;
receive, from the multi-objective policy, a candidate vehicle control action associated with the first objective and a candidate vehicle control action associated with the second objective;
determine a buffer value based on the relationship between the first objective and the second objective, wherein the buffer value includes an amount of variation of risk between the first objective and the second objective;
select a vehicle control action based on the buffer value, wherein the selected vehicle control action is the candidate vehicle control action associated with the first objective or the candidate vehicle control action associated with the second objective;
perform the selected vehicle control action to traverse a portion of the vehicle transportation network;
determine a preference indicator for each objective based on the performed vehicle control action;
determine a competence measure for each of the first objective and the second objective based on the performed vehicle control action to restrict a number of available vehicle control actions, wherein each of the competence measures of the first objective and the second objective is a learned probability of success; and

update the multi-objective policy based on the preference indicator and each of the competence measures of the first objective and the second objective.

11. The autonomous vehicle of claim 10 , wherein the preference indicator is a reward value.

12. The autonomous vehicle of claim 11 , wherein the reward value is a positive integer on a condition that the performed vehicle control action is successful.

13. The autonomous vehicle of claim 11 , wherein the reward value is a negative integer on a condition that the performed vehicle control action is overridden.

14. The autonomous vehicle of claim 10 , wherein the preference indicator is based on a user input.

15. The autonomous vehicle of claim 10 , wherein the preference indicator is a change in the buffer value.

16. The autonomous vehicle of claim 10 , wherein the preference indicator is an ordering of a plurality of objectives.

17. The autonomous vehicle of claim 10 , wherein at least the first candidate vehicle control action or the second candidate vehicle control action is based on aggregated data.

18. The autonomous vehicle of claim 10 , wherein the aggregated data is based on a geographic region, a user, a vehicle, a company, or a fleet of vehicles.

19. A method for use in a vehicle transportation network, the method comprising:
identifying, by an autonomous vehicle, a multi-objective policy for an intersection scenario-specific operational control evaluation model, wherein the multi-objective policy includes a topological graph that shows a relationship between at least two objectives, wherein an edge of the topological graph defines a constraint that restricts policies of the at least two objectives, and wherein one of the at least two objectives is associated with a passenger comfort preference that is learned based on user override data;
receiving, from the multi-objective policy for the scenario-specific operational control evaluation module instance, candidate vehicle control actions associated with each of the at least two objectives;
selecting a vehicle control action based on a buffer value, wherein the selected vehicle control action is one of the candidate vehicle control actions;
performing the selected vehicle control action to traverse a portion of the vehicle transportation network;
determining a preference indicator for each objective based on the performed vehicle control action;
determining a competence measure each of the first objective and the second objective based on the perfo

Assignees

• Nissan North America Inc
• Univ Massachusetts
• Renault Sas

Inventors

• Wray Kyle Hollins
• Witwicki Stefan
• Zilberstein Shlomo

Classifications

• B60W2050/0088

  Adaptive recalibration · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• B60W2556/50

  of positioning data, e.g. GPS [Global Positioning System] data · CPC title

• B60W50/085 Primary

  Changing the parameters of the control units, e.g. changing limit values, working points by control input · CPC title

• B60W50/0098 Primary

  Details of control systems ensuring comfort, safety or stability not otherwise provided for · CPC title