Systems and methods for motion forecasting and planning for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-12344279-B2
Application number	US-202418658674-A
Country	US
Kind code	B2
Filing date	May 8, 2024
Priority date	Nov 17, 2020
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

Systems and methods are disclosed for motion forecasting and planning for autonomous vehicles. For example, a plurality of future traffic scenarios are determined by modeling a joint distribution of actor trajectories for a plurality of actors, as opposed to an approach that models actors individually. As another example, a diversity objective is evaluated that rewards sampling of the future traffic scenarios that require distinct reactions from the autonomous vehicle. An estimated probability for the plurality of future traffic scenarios can be determined and used to generate a contingency plan for motion of the autonomous vehicle. The contingency plan can include at least one initial short-term trajectory intended for immediate action of the AV and a plurality of subsequent long-term trajectories associated with the plurality of future traffic scenarios.

First claim

What is claimed is:

1. A computer-implemented method for motion forecasting and planning, the method comprising:
(a) determining a plurality of actors within an environment of an autonomous vehicle from sensor data descriptive of the environment;
(b) determining a plurality of future motion scenarios based on the sensor data, wherein determining the plurality of future motion scenarios comprises: (i) evaluating a diversity objective that rewards sampling of the plurality of future motion scenarios that require distinct reactions from the autonomous vehicle;
(c) determining an estimated probability for the plurality of future motion scenarios; and
(d) generating a contingency plan for motion of the autonomous vehicle, wherein the contingency plan comprises a plurality of trajectories associated with the plurality of future motion scenarios, and wherein the contingency plan is generated based on the plurality of future motion scenarios and the estimated probability for the plurality of future motion scenarios.

2. The computer-implemented method of claim 1 , wherein (a) comprises processing features from corresponding map data, the corresponding map data comprising sparse map data indicative of a limited number of environmental features.

3. The computer-implemented method of claim 1 , wherein (b) comprises evaluating an energy function, the energy function indicative of a self-distance energy configured to promote diversity in behaviors of all actors associated with the plurality of future motion scenarios equally.

4. The computer-implemented method of claim 3 , wherein the energy function is further indicative of a reconstruction energy configured to preserve data reconstruction associated with the plurality of future motion scenarios.

5. The computer-implemented method of claim 1 , wherein (c) comprises employing a graph neural network (GNN) to generate a score corresponding to the estimated probability.

6. The computer-implemented method of claim 1 , wherein (d) comprises generating a single subsequent action in response to the estimated probability for the plurality of future motion scenarios.

7. The computer-implemented method of claim 1 , further comprising (e) controlling motion of the autonomous vehicle based on the contingency plan.

8. The computer-implemented method of claim 1 , wherein the contingency plan comprises at least one short-term trajectory and a plurality of subsequent long-term trajectories.

9. An autonomous vehicle (AV) control system comprising:
one or more processors; and
one or more non-transitory computer-readable media that store instructions executable by the one or more processors to cause the AV control system to perform operations, the operations comprising:
(a) determining a plurality of actors within an environment of an autonomous vehicle from sensor data descriptive of the environment;
(b) determining a plurality of future motion scenarios based on the sensor data, wherein determining the plurality of future motion scenarios comprises: (i) evaluating a diversity objective that rewards sampling of the plurality of future motion scenarios that require distinct reactions from the autonomous vehicle;
(c) determining an estimated probability for the plurality of future motion scenarios; and
(d) generating a contingency plan for motion of the autonomous vehicle, wherein the contingency plan comprises a plurality of trajectories associated with the plurality of future motion scenarios, and wherein the contingency plan is generated based on the plurality of future motion scenarios and the estimated probability for the plurality of future motion scenarios.

10. The AV control system of claim 9 , wherein (a) comprises processing features from corresponding map data, the corresponding map data comprising sparse map data indicative of a limited number of environmental features.

11. The AV control system of claim 9 , wherein (b) comprises evaluating an energy function, the energy function indicative of a self-distance energy configured to promote diversity in behaviors of all actors associated with the plurality of future motion scenarios equally.

12. The AV control system of claim 11 , wherein the energy function is further indicative of a planning diversity energy configured to promote diverse samples for subsequent motion planning tasks.

13. The AV control system of claim 9 , wherein (c) comprises employing a graph neural network (GNN) to generate a score corresponding to the estimated probability.

14. The AV control system of claim 9 , wherein (d) comprises generating a single subsequent action in response to the estimated probability for the plurality of future motion scenarios.

15. The AV control system of claim 9 , wherein the sensor data is obtained from a remote computing system.

16. The AV control system of claim 9 , wherein the contingency plan comprises at least one short-term trajectory and a plurality of subsequent long-term trajectories.

17. An autonomous vehicle comprising:
one or more processors; and
one or more non-transitory computer-readable media that store instructions executable by the one or more processors to cause the autonomous vehicle to perform operations, the operations comprising:
(a) determining a plurality of actors within an environment of the autonomous vehicle from sensor data descriptive of the environment;
(b) determining a plurality of future motion scenarios based on the sensor data, wherein determining the plurality of future motion scenarios comprises: (i) evaluating a diversity objective that rewards sampling of the plurality of future motion scenarios that require distinct reactions from the autonomous vehicle;
(c) determining an estimated probability for the plurality of future motion scenarios; and
(d) generating a contingency plan for motion of the autonomous vehicle, wherein the contingency plan comprises a plurality of trajectories associated with the plurality of future motion scenarios, and wherein the contingency plan is generated based on the plurality of future motion scenarios and the estimated probability for the plurality of future motion scenarios.

18. The autonomous vehicle of claim 17 , wherein (a) comprises processing features from corresponding map data, the corresponding map data comprising sparse map data indicative of a limited number of environmental features.

19. The autonomous vehicle of claim 17 , wherein (b) comprises evaluating an energy function, the energy function indicative of a self-distance energy configured to promote diversity in behaviors of all actors associated with the plurality of future motion scenarios equally.

20. The autonomous vehicle of claim 17 , wherein (c) comprises employing a graph neural network (GNN) to generate a score corresponding to the estimated probability.

Assignees

• Aurora Operations Inc

Inventors

• Cui Alexander Yuhao
• Sadat Abbas
• Casas Sergio
• Liao Renjie
• Urtasun Raquel

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0475

  Generative networks · CPC title