Task-informed behavior planning

Patent metadata

Field	Value
Publication number	US-12509072-B2
Application number	US-202217871628-A
Country	US
Kind code	B2
Filing date	Jul 22, 2022
Priority date	Sep 13, 2021
Publication date	Dec 30, 2025
Grant date	Dec 30, 2025

Abstract

A method for task-informed planning by a behavior planning system of a vehicle includes observing a previous trajectory of an agent within a distance from the vehicle. The method also includes predicting, by the behavior planning system, a set of potential trajectories for the agent and/or the vehicle based on observing the previous trajectory. The method further includes selecting, by the behavior planning system, a potential action from a set of potential actions associated with a task to be performed by the vehicle, each potential action being associated with a utility value based on the respective potential action and the set of potential trajectories, the selected potential action being associated with a highest utility value of respective utility values associated with the set of potential actions. The method still further includes controlling the vehicle to perform an action associated with the potential action selected by the behavior planning system.

First claim

What is claimed is:

1 . A method for task-informed planning by a behavior planning system of an autonomous vehicle, comprising:
observing, via one or more sensors associated with the autonomous vehicle, a previous trajectory of an agent that is within a distance from the autonomous vehicle; predicting, by the behavior planning system, a first set of potential trajectories for the agent and a second set of potential trajectories of the autonomous vehicle based on observing the previous trajectory; selecting, by the behavior planning system, a potential action from a set of potential actions associated with a task to be performed by the vehicle, each potential action being associated with a utility value that is a function of both an efficiency term and a safety term, each of the efficiency term and the safety term being associated with the potential action in accordance with the first set of potential trajectories and the second set of potential trajectories, the selected potential action being associated with a highest utility value of respective utility values associated with the set of potential actions; and autonomously performing, by the autonomous vehicle, an action associated with the selected potential action.

2 . The method of claim 1 , further comprising receiving a set of inputs associated with the task.

3 . The method of claim 2 , wherein:
the task is trajectory planning for the vehicle; the set of inputs includes the set of potential actions; the set of potential actions include a set of candidate trajectories of the vehicle; and the predicted set of potential trajectories includes potential trajectories of the agent.

4 . The method of claim 3 , wherein:
the behavior planning system is trained to determine the utility value based on the function that uses the efficiency term and the safety term; the efficiency term is based on a distance traveled by one candidate trajectory of the set of candidate trajectories; and the safety term is based on an expected closest distance between one candidate trajectory of the set of candidate trajectories and the set of potential trajectories.

5 . The method of claim 1 , wherein:
the task is warning generation at the vehicle; the set of potential actions include a first potential action associated with generating a warning and a second potential action associated with not generating the warning; and the predicted set of potential trajectories include a set of potential agent trajectories a set of potential vehicle trajectories.

6 . The method of claim 5 , wherein the warning term is associated with a likelihood of a collision between each potential agent trajectory of the set of potential agent trajectories and each potential vehicle trajectory of the set of potential vehicle trajectories.

7 . The method of claim 1 , further comprising:
training the behavior planning system to predict the set of potential trajectories by minimizing a loss between a set of potential training trajectories and a ground truth trajectory; and training the behavior planning system to select the potential action by minimizing a cross entropy between a decision utility and a ground truth decision.

8 . An apparatus for task-informed planning by a behavior planning system of a vehicle, comprising:
at least one processor; and at least one memory coupled with the at least one processor and storing instructions operable, when executed by the at least one processor, to cause the apparatus:
observe, via one or more sensors associated with the autonomous vehicle, a previous trajectory of an agent that is within a distance from the autonomous vehicle;
predict, by the behavior planning system, a first set of potential trajectories for the agent and a second set of potential trajectories of the autonomous vehicle based on observing the previous trajectory;
select, by the behavior planning system, a potential action from a set of potential actions associated with a task to be performed by the vehicle, each potential action being associated with a utility value that is a function of both an efficiency term and a safety term, each of the efficiency term and the safety term being associated with the potential action in accordance with the first set of potential trajectories and the second set of potential trajectories, the selected potential action being associated with a highest utility value of respective utility values associated with the set of potential actions; and
autonomously perform, by the autonomous vehicle, an action associated with the selected potential action.

9 . The apparatus of claim 8 , wherein execution of the instructions further cause the apparatus to receive a set of inputs associated with the task.

10 . The apparatus of claim 9 , wherein:
the task is trajectory planning for the vehicle; the set of inputs includes the set of potential actions; the set of potential actions include a set of candidate trajectories of the vehicle; and the predicted set of potential trajectories includes potential trajectories of the agent.

11 . The apparatus of claim 10 , wherein:
the behavior planning system is trained to determine the utility value based on the function that uses the efficiency term and the safety term; the efficiency term is based on a distance traveled by one candidate trajectory of the set of candidate trajectories; and the safety term is based on an expected closest distance between one candidate trajectory of the set of candidate trajectories and the set of potential trajectories.

12 . The apparatus of claim 8 , wherein:
the task is warning generation at the vehicle; the set of potential actions include a first potential action associated with generating a warning and a second potential action associated with not generating the warning; and the predicted set of potential trajectories include a set of potential agent trajectories a set of potential vehicle trajectories.

13 . The apparatus of claim 12 , wherein the warning term is associated with a likelihood of a collision between each potential agent trajectory of the set of potential agent trajectories and each potential vehicle trajectory of the set of potential vehicle trajectories.

14 . The apparatus of claim 8 , wherein execution of the instructions further cause the apparatus to:
train the behavior planning system to predict the set of potential trajectories by minimizing a loss between a set of potential training trajectories and a ground truth trajectory; and train the behavior planning system to select the potential action by minimizing a cross entropy between a decision utility and a ground truth decision.

15 . A non-transitory computer-readable medium having program code recorded thereon for task-informed planning by a behavior planning system of a vehicle, the program code executed by at least one processor and comprising:
program code to observe, via one or more sensors associated with the autonomous vehicle, a previous trajectory of an agent that is within a distance from the autonomous vehicle; program code to predict, by the behavior planning system, a first set of potential trajectories for the agent and a second set of potential trajectories of the autonomous vehicle based on observing the previous trajectory; program code to select, by the behavior planning system, a potential action from a set of potential actions associated with a task to be performed by the vehicle, each potential action being associated with a utility value that is a function of both an efficiency term and a safety term, each of the efficiency term and the safety t

Assignees

• Toyota Res Inst Inc
• Massachusetts Institue Of Tech
• Toyota Motor Co Ltd
• Massachusetts Inst Technology

Inventors

• Huang Xin
• Rosman Guy
• Jasour Ashkan Mohammadzadeh
• Mcgill Jr Stephen G
• Leonard John J
• Williams Brian C

Classifications

• B60W60/0027

  using trajectory prediction for other traffic participants · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W2554/802

  Longitudinal distance · CPC title

• B60W30/0956 Primary

  the prediction being responsive to traffic or environmental parameters · CPC title