Online state space refinement for competence-aware systems

Patent metadata

Field	Value
Publication number	US-12344280-B2
Application number	US-202217828815-A
Country	US
Kind code	B2
Filing date	May 31, 2022
Priority date	May 31, 2022
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

A first method includes detecting, based on sensor data, an environment state; selecting an action based on the environment state; determining an autonomy level associated with the environment state and the action; and performing the action according to the autonomy level. The autonomy level can be selected based at least on an autonomy model and a feedback model. A second method includes calculating, by solving an extended Stochastic Shortest Path (SSP) problem, a policy for solving a task. The policy can map environment states and autonomy levels to actions and autonomy levels. Calculating the policy can include generating plans that operate across multiple levels of autonomy.

First claim

What is claimed is:

1. A method of autonomous driving by an autonomous vehicle (AV), comprising:
detecting, based on sensor data, an environment state;
selecting an action based on the environment state;
identifying a current set of indiscriminate states;
identifying a discriminator from the current set of indiscriminate states;
training a feedback model for the discriminator;
determining an autonomy level associated with the environment state and the action, wherein the autonomy level is selected based at least on an autonomy model and the feedback model; and
performing the action according to the autonomy level.

2. The method of claim 1 , wherein the autonomy level is selected from a set comprising a first autonomy level indicating “no autonomy”, a second autonomy level indicating “verified autonomy”, a third autonomy level indicating “supervised autonomy”, and a fourth autonomy level indicating “unsupervised autonomy”.

3. The method of claim 2 , wherein the autonomy level is the second autonomy level indicating “verified autonomy”, and wherein performing the action according to the autonomy level comprising:
receiving, for the action, an approval feedback signal or a disapproval feedback signal.

4. The method of claim 3 , wherein performing the action according to the autonomy level comprising:
querying for the approval feedback signal prior to receiving the approval feedback signal.

5. The method of claim 2 ,
wherein the autonomy level is the third autonomy level indicating “supervised autonomy”, and
wherein performing the action according to the autonomy level comprising:
determining that the AV is being monitored by a human before performing the action.

6. The method of claim 3 ,
wherein the approval feedback signal is received, and
wherein performing the action according to the autonomy level comprising:
determining that the AV is being monitored in response to determining that the AV is being monitored, performing the action;
receiving an override signal; and
in response to receiving the override signal, stopping the action and switching to a manual operation mode of the AV.

7. The method of claim 1 , wherein performing the action according to the autonomy level comprising:
determining, based on the autonomy level, whether to request approval from a human for the action before performing the action.

8. The method of claim 1 ,
wherein the autonomy level is an “no autonomy” such that the AV is not allowed to perform autonomous actions, and
wherein performing the action according to the autonomy level comprising:
enabling the AV to be manually controlled by a human.

9. The method of claim 1 ,
wherein the autonomy model comprises a utility model and an autonomy profile,
wherein utility model describes a utility of performing a first action in a first autonomy level with respect to a first environment state given that the AV transitioned from a second autonomy level, and
wherein the autonomy profile maps respective environment states to respective actions and prescribing constraints on allowed levels of autonomy for particular environment states.

10. The method of claim 1 , further comprising:
updating at least one of an autonomy profile, a feedback profile, or a human transition function in response to the performing the action.

11. A system for autonomous comprising:
a memory; and
a processor, the processor configured to execute instructions stored in the memory to:
calculate, by solving an extended Stochastic Shortest Path (SSP) problem, a policy for solving a task,
identify a discriminator from a current set of indiscriminate states; and
train a feedback model for the discriminator;
wherein the policy maps environment states and autonomy levels to actions and autonomy levels, and
wherein to calculate the policy comprises to:
generate plans that operate across multiple levels of autonomy.

12. The system of claim 11 , wherein to generate plans that operate across the multiple levels of autonomy comprises to:
generate plans subject to constraints on allowed levels of autonomy in respective states.

13. The system of claim 12 , wherein a constraint maps a state and an action to a subset of levels of autonomy.

14. The system of claim 11 , wherein the instructions further comprise instructions to:
update the feedback model representing a first probability that the system receives a first signal when performing a first action at a second autonomy level given that the system is in a first state and the system transitioned from a first autonomy level.

15. The system of claim 11 , wherein the instructions further comprise instructions to:
update a human state transition function representing a second probability of a human transitioning to a second state of an environment model given that the system selected to perform a second action in a first state and the human took manual control.

16. The system of claim 11 , wherein the instructions further comprise instructions to:
update an autonomy profile, wherein the autonomy profile defines a set of acceptable autonomy levels given a current state and an action to be performed next.

17. A method for autonomous driving, comprising:
calculating, by solving an extended Stochastic Shortest Path (SSP) problem, a policy for solving a task,
identifying a discriminator from a current set of indiscriminate states; and
training a feedback model for the discriminator;
wherein the policy maps environment states and autonomy levels to actions and autonomy levels, and
wherein calculating the policy comprising:
generating plans that operate across multiple levels of autonomy.

18. The method of claim 17 , wherein generating plans that operate across the multiple levels of autonomy comprising:
generating plans subject to constraints on allowed levels of autonomy in respective states.

19. The method of claim 18 , wherein a constraint maps a state and an action to a subset of levels of autonomy.

20. The method of claim 17 , further comprising:
updating the feedback model representing a first probability of receiving a first signal when performing a first action at a second autonomy level given that an agent is in a first state and the agent transitioned from a first autonomy level;
updating a human state transition function representing a second probability of a human transitioning to a second state of an environment model given that the agent selected to perform a second action in a third state and the human took manual control; and
updating an autonomy profile, wherein the autonomy profile defines a set of acceptable autonomy levels given a current state and an action to be performed next.

Assignees

• Nissan North America Inc
• Univ Massachusetts

Inventors

• Basich Connor
• Wray Kyle Hollins
• Witwicki Stefan
• Zilberstein Shlomo

Classifications

• B60W2050/0028

  Mathematical models, e.g. for simulation · CPC title

• B60W50/00

  Details of control systems for road vehicle drive control not related to the control of a particular sub-unit {, e.g. process diagnostic or vehicle driver interfaces} · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• B60W2554/00

  Input parameters relating to objects · CPC title

• B60W40/04

  Traffic conditions · CPC title