Perception simulation for improved autonomous vehicle control

Patent metadata

Field	Value
Publication number	US-12228472-B2
Application number	US-202418424318-A
Country	US
Kind code	B2
Filing date	Jan 26, 2024
Priority date	May 18, 2017
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

A system and method for real world autonomous vehicle perception simulation are disclosed. A particular embodiment includes: configuring a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters; using the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and providing the simulated perception data to a motion planning system for the autonomous vehicle.

First claim

What is claimed is:

1. A system comprising:
a data processor; and
a perception simulation module, executable by the data processor, the perception simulation module being configured to:
configure a sensor noise modeling module to produce simulated sensor errors or noise data by comparing calibrated sensor data from one or more of a plurality of sensors with ground truth data;
use the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of the plurality of sensors; and
provide the simulated perception data to a motion planning system for an autonomous vehicle.

2. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating a false positive or false negative object detection.

3. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in vehicle localization.

4. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in lane detection.

5. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in object type classification.

6. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in object location.

7. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in data produced by an image processing module.

8. A method comprising:
configuring a sensor noise modeling module to produce simulated sensor errors or noise data by comparing calibrated sensor data from one or more of a plurality of sensors with ground truth data;
using the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of the plurality of sensors; and
providing the simulated perception data to a motion planning system for an autonomous vehicle.

9. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating a false positive or false negative result produced by a perception processing module.

10. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating noise produced by a perception sensor device.

11. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating noise in perception sensor devices and sensor data processing modules over a pre-determined time period.

12. The method of claim 8 including producing simulated perception data that corresponds to actual or simulated ground truth data.

13. The method of claim 8 including configuring or training the motion planning system for the autonomous vehicle or a vehicle simulation system.

14. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating errors corresponding to physical constraints of a perception sensor used with the autonomous vehicle.

15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
configure a sensor noise modeling module to produce simulated sensor errors or noise data by comparing calibrated sensor data from one or more of a plurality of sensors with ground truth data;
use the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of the plurality of sensors; and
provide the simulated perception data to a motion planning system for an autonomous vehicle.

16. The non-transitory machine-useable storage medium of claim 15 wherein the plurality of sensors includes: image generating devices, cameras, distance measuring devices, light amplification by stimulated emission of radiation (laser) devices or light detection and ranging (LIDAR) devices, global positioning systems (GPS) devices, sound navigation and ranging (sonar) devices, or radio detection and ranging (radar) devices.

17. The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating a false positive or false negative object classification.

18. The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in vehicle trajectory.

19. The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in roadway detection.

20. The non-transitory machine-useable storage medium of claim 15 being configured to perform a calibration phase based on a comparison of real world perception data and ground truth data.

Assignees

• Tusimple Inc

Inventors

• Sun Xing
• Lin Wutu
• Zhao Yufei
• Liu Liu

Classifications

• B60W30/00

  Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units · CPC title

• B60W2050/0215

  Sensor drifts or sensor failures · CPC title

• G05D1/0257

  using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931) · CPC title

• G05D1/0223

  involving speed control of the vehicle (vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00) · CPC title

• G05D1/0221

  involving a learning process · CPC title