System and method for real world autonomous vehicle trajectory simulation

Patent metadata

Field	Value
Publication number	US-10739775-B2
Application number	US-201715796765-A
Country	US
Kind code	B2
Filing date	Oct 28, 2017
Priority date	Oct 28, 2017
Publication date	Aug 11, 2020
Grant date	Aug 11, 2020

Abstract

A system and method for real world autonomous vehicle trajectory simulation are disclosed. A particular embodiment includes: receiving training data from a real world data collection system; obtaining ground truth data corresponding to the training data; performing a training phase to train a plurality of trajectory prediction models; and performing a simulation or operational phase to generate a vicinal scenario for each simulated vehicle in an iteration of a simulation, the vicinal scenarios corresponding to different locations, traffic patterns, or environmental conditions being simulated, provide vehicle intention data corresponding to a data representation of various types of simulated vehicle or driver intentions, generate a trajectory corresponding to perception data and the vehicle intention data, execute at least one of the plurality of trained trajectory prediction models to generate a distribution of predicted vehicle trajectories for each of a plurality of simulated vehicles of the simulation based on the vicinal scenario and the vehicle intention data, select at least one vehicle trajectory from the distribution based on pre-defined criteria, and update a state and trajectory of each of the plurality of simulated vehicles based on the selected vehicle trajectory from the distribution.

First claim

What is claimed is:

1. A system comprising:
a data processor in a computing system;
a data collection system, executable by the data processor of the computing system, to collect training data comprising perception or sensor data and ground truth data, and perform a training phase to train a plurality of trajectory prediction models with the training data to produce a plurality of trained trajectory prediction models, the data collection system comprising sensor devices, the sensor devices comprising at least one image generating device, the sensor devices, installed at various traffic locations or installed on moving test vehicles, collecting perception or sensor data from the various traffic locations or from the moving test vehicles, the plurality of trained trajectory prediction models comprising at least one trained trajectory prediction model configured to model a variable level of simulated driver aggressiveness, the collected training data being transferred to the data collection system;
a vicinal scene data generator to generate, by the data processor, a different vicinal scenario for each simulated vehicle in an iteration of a simulation;
a memory for storage of vehicle intention data representing simulated vehicle or driver intentions for a plurality of different vehicle actions and behaviors; and
a trajectory simulation module, executable by the data processor of the computing system, the trajectory simulation module being configured to perform a simulation or operational phase to:
generate, by the data processor, a trajectory corresponding to the collected perception data and the vehicle intention data, execute, by the data processor, at least one of the plurality of trained trajectory prediction models to generate a predicted simulated vehicle trajectory for each of a plurality of simulated vehicles of the simulation based on a corresponding generated vicinal scenario and the vehicle intention data, the predicted simulated vehicle trajectory comprising data indicative of a degree of likelihood or probability that a particular simulated vehicle will traverse the corresponding predicted simulated vehicle trajectory, use the predicted simulated vehicle trajectory, and update, by the data processor, a state and trajectory of each of the plurality of simulated vehicles based on the predicted simulated vehicle trajectory, the plurality of trained trajectory prediction models being used for configuring a control system in an autonomous vehicle.

2. The system of claim 1 wherein the trajectory simulation operation comprises machine learnable components.

3. The system of claim 1 wherein the trajectory simulation operation is performed over multiple iterations.

4. The system of claim 1 wherein the trajectory simulation operation being further configured to receive an array of sensor information gathered at the various traffic locations by the data collection system.

5. The system of claim 1 wherein the different vicinal scenarios correspond to different locations, traffic patterns, or environmental conditions being simulated.

6. The system of claim 1 wherein the trajectory simulation module being further configured to execute, by the data processor, at least one of the plurality of trained trajectory prediction models to generate a distribution of predicted simulated vehicle trajectories for each of a plurality of simulated vehicles of the simulation based on a corresponding generated vicinal scenario and the vehicle intention data, the distribution of predicted simulated vehicle trajectories comprising data indicative of a degree of likelihood or probability that a particular simulated vehicle will traverse a corresponding trajectory of the distribution of predicted simulated vehicle trajectories, select at least one simulated vehicle trajectory from the distribution of predicted simulated vehicle trajectories based on pre-defined criteria, and update, by the data processor, a state and trajectory of each of the plurality of simulated vehicles based on the selected simulated vehicle trajectory from the distribution of predicted simulated vehicle trajectories.

7. The system of claim 1 wherein the variable level of simulated driver aggressiveness is modeled from behaviors of human drivers collected by the data collection system.

8. A method comprising:
receiving training data, by a data processor in a computing system, from a data collection system, the data collection system comprising sensor devices, the sensor devices comprising at least one image generating device, the sensor devices, installed at various traffic locations or installed on moving test vehicles, collecting perception or sensor data from the various traffic locations or from the moving test vehicles;
obtaining ground truth data corresponding to the training data;
transferring the collected training data to the data collection system;
using the data collection system, executed by the data processor of the computing system, and the training data to train a plurality of trajectory prediction models to produce a plurality of trained trajectory prediction models, the plurality of trained trajectory prediction models comprising at least one trained trajectory prediction model configured to model a variable level of simulated driver aggressiveness; and
performing, by the data processor, a simulation or operational phase to generate a different vicinal scenario for each simulated vehicle in an iteration of a simulation, provide vehicle intention data corresponding to a data representation of various types of simulated vehicle or driver intentions for a plurality of different vehicle actions and behaviors, generate, by the data processor, a trajectory corresponding to the collected perception data and the vehicle intention data, execute, by the data processor, at least one of the plurality of trained trajectory prediction models to generate a predicted simulated vehicle trajectory for each of a plurality of simulated vehicles of the simulation based on a corresponding generated vicinal scenario and the vehicle intention data, the predicted simulated vehicle trajectory comprising data indicative of a degree of likelihood or probability that a particular simulated vehicle will traverse the corresponding predicted simulated vehicle trajectory, use the predicted simulated vehicle trajectory, and update, by the data processor, a state and trajectory of each of the plurality of simulated vehicles based on the predicted simulated vehicle trajectory, the plurality of trained trajectory prediction models being used for configuring a control system in an autonomous vehicle.

9. The method of claim 8 wherein the method is machine learnable.

10. The method of claim 8 wherein the method is performed over multiple iterations.

11. The method of claim 8 comprising receiving an array of sensor information gathered at the various traffic locations by the data collection system.

12. The method of claim 8 wherein the different vicinal scenarios correspond to different locations, traffic patterns, or environmental conditions being simulated.

13. The method of claim 8 wherein the simulation or operational phase further to execute, by the data processor, at least one of the plurality of trained trajectory prediction models to generate a distribution of predicted simulated vehicle trajectories for each of a plurality of simulated vehicles of the simulation based on a corresponding generated vicinal scenario and the vehicle intention data, the distribution of predicted simulated vehicle trajectories comprising data indicative of a degree of likelihood or probability that a particular simulated vehicle will traverse a c

Assignees

• Tusimple Inc

Inventors

• Sun Xing
• Lin Wutu
• Liu Liu
• Ma Kai-Chieh
• Xuan Zijie
• Zhao Yufei

Classifications

• B60W60/00276 Primary

  for two or more other traffic participants · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G05D1/646

  Following a predefined trajectory, e.g. a line marked on the floor or a flight path · CPC title

• G05D2101/15

  using machine learning, e.g. neural networks · CPC title