System and method for real world autonomous vehicle trajectory simulation

Patent metadata

Field	Value
Publication number	US-11853072-B2
Application number	US-202217901736-A
Country	US
Kind code	B2
Filing date	Sep 1, 2022
Priority date	Oct 28, 2017
Publication date	Dec 26, 2023
Grant date	Dec 26, 2023

Abstract

A system and method for real world autonomous vehicle trajectory simulation may include: receiving training data from a data collection system; obtaining ground truth data corresponding to the training data; performing a training phase to train a plurality of trajectory prediction models; and performing a simulation or operational phase to generate a vicinal scenario for each simulated vehicle in an iteration of a simulation. Vicinal scenarios may correspond to different locations, traffic patterns, or environmental conditions being simulated. Vehicle intention data corresponding to a data representation of various types of simulated vehicle or driver intentions.

First claim

What is claimed is:

1. A system comprising:
a data processor;
a data collection system interface for receiving perception or sensor data collected from a data collection system, the perception or sensor data including data from at least one image generating device;
a memory for storage of a plurality of trained trajectory prediction models, the plurality of trained trajectory prediction models having been trained using real world training data and simulated driver behaviors;
a trajectory generation module, executable by the data processor, the trajectory generation module being configured to:
generate, by the data processor, a trajectory for an autonomous vehicle, the trajectory corresponding to the perception or sensor data and autonomous vehicle intention data;
execute, by the data processor, at least one of the plurality of trained trajectory prediction models to generate a distribution of predicted vehicle trajectories for each of a plurality of vehicles proximate to the autonomous vehicle based on the perception or sensor data and the autonomous vehicle intention data;
select, by the data processor, at least one predicted vehicle trajectory from the distribution for each proximate vehicle based on pre-defined criteria; and
modify, by the data processor, the trajectory for the autonomous vehicle to avoid the at least one predicted vehicle trajectory for each proximate vehicle; and

an autonomous vehicle control system for the autonomous vehicle configured to cause the autonomous vehicle to traverse the modified trajectory for the autonomous vehicle.

2. The system of claim 1 wherein the data collection system is further configured to perform a training phase to train the plurality of trajectory prediction models with training data to produce the plurality of trained trajectory prediction models.

3. The system of claim 1 wherein the plurality of trained trajectory prediction models comprising at least one trained trajectory prediction model configured to model a variable level of driver aggressiveness.

4. The system of claim 1 wherein the memory is further configured to store the perception or sensor data collected from the data collection system.

5. The system of claim 1 wherein the perception or sensor data includes traffic image data, vehicle image data, roadway data, environmental data, distance data from LIDAR devices, and distance data from radar devices, the data collection system being installed on the autonomous vehicle.

6. The system of claim 1 wherein the autonomous vehicle control system being configured to control a direction and speed of the autonomous vehicle to direct the autonomous vehicle to follow the modified trajectory for the autonomous vehicle.

7. A method comprising:
receiving, by a data processor, perception or sensor data collected from a data collection system, the perception or sensor data including data from at least one image generating device;
storing, by a memory device, a plurality of trained trajectory prediction models, the plurality of trained trajectory prediction models having been trained using real world training data and simulated driver behaviors;
generating, by the data processor, a trajectory for an autonomous vehicle, the trajectory corresponding to the perception or sensor data and autonomous vehicle intention data;
executing, by the data processor, at least one of the plurality of trained trajectory prediction models to generate a distribution of predicted vehicle trajectories for each of a plurality of vehicles proximate to the autonomous vehicle based on the perception or sensor data and the autonomous vehicle intention data;
selecting, by the data processor, at least one predicted vehicle trajectory from the distribution for each proximate vehicle based on pre-defined criteria;
modifying, by the data processor, the trajectory for the autonomous vehicle to avoid the at least one predicted vehicle trajectory for each proximate vehicle; and
using an autonomous vehicle control system for the autonomous vehicle to cause the autonomous vehicle to traverse the modified trajectory for the autonomous vehicle.

8. The method of claim 7 further including performing a training phase to train the plurality of trajectory prediction models with training data to produce the plurality of trained trajectory prediction models.

9. The method of claim 7 wherein the plurality of trained trajectory prediction models comprising at least one trained trajectory prediction model configured to model a variable level of driver conservatism.

10. The method of claim 7 wherein the distribution of predicted vehicle trajectories includes data indicative of a degree of likelihood or probability that a particular proximate vehicle will traverse a corresponding trajectory of the distribution.

11. The method of claim 7 , wherein the pre-defined criteria including a maximal or minimal likelihood rating or probability value, conformity with pre-defined safety parameters, conformity with pre-defined economy parameters, and conformity with pre-defined timing or distance parameters.

12. The method of claim 7 , wherein the plurality of trained trajectory prediction models includes modeling of a transport time delay between a stimulus and a driver's control response.

13. The method of claim 7 wherein the plurality of trained trajectory prediction models includes a speed control model and a cornering aggressiveness model.

14. The method of claim 7 including controlling a direction and speed of the autonomous vehicle to direct the autonomous vehicle to follow the modified trajectory for the autonomous vehicle.

15. A non-transitory machine-readable storage medium embodying instructions which, when executed by a data processor, cause the data processor to:
receive perception or sensor data collected from a data collection system, the perception or sensor data including data from at least one image generating device;
store in a memory device a plurality of trained trajectory prediction models, the plurality of trained trajectory prediction models having been trained using real world training data and simulated driver behaviors;
generate a trajectory for an autonomous vehicle, the trajectory corresponding to the perception or sensor data and autonomous vehicle intention data;
execute at least one of the plurality of trained trajectory prediction models to generate a distribution of predicted vehicle trajectories for each of a plurality of vehicles proximate to the autonomous vehicle based on the perception or sensor data and the autonomous vehicle intention data;
select at least one predicted vehicle trajectory from the distribution for each proximate vehicle based on pre-defined criteria;
modify the trajectory for the autonomous vehicle to avoid the at least one predicted vehicle trajectory for each proximate vehicle; and
use an autonomous vehicle control system for the autonomous vehicle to cause the autonomous vehicle to traverse the modified trajectory for the autonomous vehicle.

16. The non-transitory machine-readable storage medium of claim 15 wherein the instructions include machine learnable components.

17. The non-transitory machine-readable storage medium of claim 15 wherein the instructions being further configured to perform a training phase to train the plurality of trajectory prediction models with training data to produce the plurality of trained trajectory prediction models.

18. The non-transitory machine-readable storage medium of claim 15 wherein the plurality of trained trajectory prediction models comprisi

Assignees

• Tusimple Inc

Inventors

• Sun Xing
• Lin Wutu
• Liu Liu
• Ma Kai-Chieh
• Xuan Zijie
• Zhao Yufei

Classifications

• B60W60/00276 Primary

  for two or more other traffic participants · CPC title

• G06N3/09

  Supervised learning · CPC title

• G05D1/646

  Following a predefined trajectory, e.g. a line marked on the floor or a flight path · CPC title

• G05D2101/15

  using machine learning, e.g. neural networks · CPC title

• G05D1/0221 Primary

  involving a learning process · CPC title