Trajectory prediction from precomputed or dynamically generated bank of trajectories

Patent metadata

Field	Value
Publication number	US-11858508-B2
Application number	US-202218079399-A
Country	US
Kind code	B2
Filing date	Dec 12, 2022
Priority date	Nov 7, 2019
Publication date	Jan 2, 2024
Grant date	Jan 2, 2024

Abstract

Among other things, techniques are described for predicting how an agent (e.g., a vehicle, bicycle, pedestrian, etc.) will move in an environment based on prior movement, the road network, the surrounding objects and/or other relevant environmental factors. One trajectory prediction technique involves generating a probability map for an agent's movement. Another trajectory prediction technique involves generating a trajectory lattice, for an agent's movement. In addition, a different trajectory prediction technique involves multi-modal regression where a classifier (e.g., a neural network) is trained to classify the probability of a number of (learned) modes such that each model produces a trajectory based on the current input.

First claim

What is claimed is:

1. A computer implemented method comprising:
receiving, by one or more processors, location data and past trajectory data for one or more objects detected by one or more sensors;
determining, by the one or more processors, a set of features for the one or more objects based on the location data and the past trajectory data;
combining, by the one or more processors, the set of features, a trajectory traveled by an agent, and motion data of the agent to form a concatenated data set;
generating, by the one or more processors, a trajectory lattice corresponding to the agent based on the concatenated data set, wherein the trajectory lattice is a plurality of possible trajectories with a respective probabilities;
determining, by the one or more processors, one or more predicted trajectories for the agent based on the trajectory lattice;
generating, by the one or more processors, one or more driving commands for a vehicle based on the one or more predicted trajectories; and
operating, by the one or more processors, the vehicle based on the one or more driving commands.

2. The method of claim 1 , comprising dynamically generating the trajectory lattice based on a state of the agent.

3. The method of claim 1 , comprising removing trajectories from the trajectory lattice that the agent cannot travel based on road rules data or road marking data to obtain an updated trajectory lattice, and determining predicted trajectories based on the updated trajectory lattice.

4. The method of claim 1 , wherein receiving the past trajectory data comprises receiving, by the one or more processors, a trajectory of each object of the one or more object for a past time interval.

5. The method of claim 1 , wherein receiving the location data and the past trajectory data comprises receiving, by the one or more processors, an image including the location data for the one or more objects and the past trajectory data for the one or more objects, wherein the past trajectory data is color coded to indicate a corresponding past trajectory for each object of the one or more objects.

6. The method of claim 5 , wherein determining the set of features for the one or more objects, comprises:
inputting the image into a classifier; and
receiving a plurality of features for the image from the classifier.

7. The method of claim 1 , wherein generating the trajectory lattice comprises generating a data structure with a plurality of fields for each trajectory in the trajectory lattice comprising a coordinate field and a probability field.

8. The method of claim 1 , further comprising:
receiving training location data and training past trajectory data for one or more training objects;
determining a set of training features for the one or more training objects based on the training location data and the training past trajectory data;
combining the set of training features with training motion data of an agent to form a training concatenated data set;
generating, based on the training concatenated data set, a training trajectory lattice, wherein the training trajectory lattice comprises a training plurality of predicted trajectories, and wherein each of the training plurality of predicted trajectories is assigned a training probability of a training agent traveling a respective trajectory;
determining, based on the training trajectory lattice, one or more training trajectories for the training agent;
comparing the one or more training trajectories with a known trajectory of the training agent; and
updating weights of a neural network according to the comparing, wherein the neural network outputs the trajectory lattice.

9. The method of claim 8 , wherein updating the weights of the neural network according to the comparing comprises propagating a difference between each of the one or more training trajectories and the known trajectory through the neural network.

10. A system, comprising:
at least one hardware processor; and
at least one computer-readable medium storing computer-executable instructions;
wherein the computer-executable instructions, when executed by the at least one hardware processor, cause the system to:
receive location data and past trajectory data for one or more objects detected by one or more sensors;
determine a set of features for the one or more objects based on the location data and the past trajectory data;
combine the set of features, a trajectory traveled by an agent, and motion data of the agent to form a concatenated data set;
generate a trajectory lattice corresponding to the agent based on the concatenated data set, wherein the trajectory lattice is a plurality of possible trajectories with a respective probabilities;
determine one or more predicted trajectories for the agent based on the trajectory lattice;
generate one or more driving commands for a vehicle based on the one or more predicted trajectories; and

operate the vehicle based on the one or more driving commands.

11. The system of claim 10 , wherein the computer-executable instructions cause the system to dynamically generating the trajectory lattice based on a state of the agent.

12. The system of claim 10 , wherein the computer-executable instructions cause the system to remove trajectories from the trajectory lattice that the agent cannot travel based on road rules data or road marking data to obtain an updated trajectory lattice, and determine predicted trajectories based on the updated trajectory lattice.

13. The system of claim 10 , wherein the computer-executable instructions cause the system to receive the past trajectory data, comprising receiving a trajectory of each object of the one or more object for a past time interval.

14. The system of claim 10 , wherein the computer-executable instructions cause the system to receive the location data and the past trajectory data, comprising receiving an image including the location data for the one or more objects and the past trajectory data for the one or more objects, wherein the past trajectory data is color coded to indicate a corresponding past trajectory for each object of the one or more objects.

15. The system of claim 14 , wherein the computer-executable instructions cause the system to determine the set of features for the one or more objects, comprising:
inputting the image into a classifier; and
receiving a plurality of features for the image from the classifier.

16. The system of claim 10 , wherein the computer-executable instructions cause the system to generate the trajectory lattice comprising generating a data structure with a plurality of fields for each trajectory in the trajectory lattice comprising a coordinate field and a probability field.

17. A non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors, the one or more programs including instructions which, when executed by the one or more processors, cause the one or more processors to:
receive location data and past trajectory data for one or more objects detected by one or more sensors;
determine a set of features for the one or more objects based on the location data and the past trajectory data;
combine the set of features, a trajectory traveled by an agent, and motion data of the agent to form a concatenated data set;
generate a trajectory lattice corresponding to the agent based on the concatenated data set, wherein the trajectory lattice is a plurality of possible trajectories with a respective probabilities;
determine one or more predicted trajectories for the agent based on the tr

Assignees

• Motional Ad Llc

Inventors

• Phan Tung Minh
• Wolff Eric
• Frazzoli Emilio
• Grigore Elena Corina
• Boulton Freddy

Classifications

• B60W30/0956 Primary

  the prediction being responsive to traffic or environmental parameters · CPC title

• G01S7/4814

  of transmitters alone · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G01S17/931 Primary

  of land vehicles · CPC title

• G01S19/01

  Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO · CPC title