System and method for proximate vehicle intention prediction for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11948082-B2
Application number	US-202117401781-A
Country	US
Kind code	B2
Filing date	Aug 13, 2021
Priority date	May 31, 2018
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

A system and method for proximate vehicle intention prediction for autonomous vehicles are disclosed. A particular embodiment is configured to: receive perception data associated with a host vehicle; extract features from the perception data to detect a proximate vehicle in the vicinity of the host vehicle; generate a trajectory of the detected proximate vehicle based on the perception data; use a trained intention prediction model to generate a predicted intention of the detected proximate vehicle based on the perception data and the trajectory of the detected proximate vehicle; use the predicted intention of the detected proximate vehicle to generate a predicted trajectory of the detected proximate vehicle; and output the predicted intention and predicted trajectory for the detected proximate vehicle to another subsystem.

First claim

What is claimed is:

1. A system, comprising:
a processor; and
a memory for storing instructions, wherein the instructions, when executed by the processor, causes the processor to:
receive perception data associated with a host vehicle;
extract features from the perception data;
detect a proximate vehicle in a vicinity of the host vehicle using the extracted features;
generate a trajectory of the detected proximate vehicle based on the perception data;
generate a predicted intention of the detected proximate vehicle using the perception data, the trajectory of the detected proximate vehicle, and an intention prediction model;
modify the predicted intention of the detected proximate vehicle using a Hidden Markov Model (HMM); and
generate a predicted trajectory of the detected proximate vehicle using the modified predicted intention.

2. The system of claim 1 , wherein the processor is further configured to:
output the modified predicted intention and the predicted trajectory of the detected proximate vehicle to a motion planner of the host vehicle.

3. The system of claim 1 , wherein the perception data comprises data received from at least one of: a camera, an image capture device, an inertial measurement unit (IMU), a global positioning system (GPS) transceiver, a radio detection and ranging (RADAR) device, a laser range finder, a light detection and ranging (LIDAR) device, or a sound navigation and ranging (sonar) device.

4. The system of claim 1 , wherein the system is configured to train the intention prediction model using training data.

5. The system of claim 1 , wherein the processor is configured to detect the proximate vehicle using semantic segmentation on an image frame sequence from the perception data.

6. The system of claim 1 , wherein the processor is configured to generate the trajectory of the detected proximate vehicle using aggregating perception data for the detected proximate vehicle across multiple image frames.

7. A method, comprising:
receiving perception data associated with a host vehicle;
extracting features from the perception data;
detecting a proximate vehicle in a vicinity of the host vehicle using the extracted features;
generating a trajectory of the detected proximate vehicle based on the perception data;
generating a predicted intention of the detected proximate vehicle using the perception data, the trajectory of the detected proximate vehicle, and an intention prediction model;
modifying the predicted intention of the detected proximate vehicle using a Hidden Markov Model (HMM); and
generating a predicted trajectory of the detected proximate vehicle using the modified predicted intention.

8. The method of claim 7 , further comprising:
outputting the modified predicted intention and the predicted trajectory of the detected proximate vehicle to a vehicle subsystem of the host vehicle.

9. The method of claim 7 , wherein the generating the predicted intention of the detected proximate vehicle comprises:
generating a distribution of probabilistic maneuvers associated with the detected proximate vehicle using the perception data and the trajectory of the detected proximate vehicle.

10. The method of claim 9 , further comprising:
obtaining a filtered distribution of the probabilistic maneuvers associated with the detected proximate vehicle by applying a Bayesian filter to the distribution of the probabilistic maneuvers associated with the detected proximate vehicle.

11. The method of claim 10 , wherein the Bayesian filter is parameterized using driving maneuver pattern distributions.

12. The method of claim 10 , further comprising:
modifying the filtered distribution of the probabilistic maneuvers associated with the detected proximate vehicle using a second HMM.

13. The method of claim 7 , further comprising:
updating the HMM using observed maneuvers associated with the detected proximate vehicle.

14. The method of claim 7 , wherein the trajectory of the detected proximate vehicle is a smoothed trajectory of the detected proximate vehicle.

15. The method of claim 14 , wherein the smoothed trajectory is obtained using temporal smoothing.

16. A non-transitory computer readable storage medium comprising executable instructions that, when executed by at least one processor, cause the at least one processor to perform operations, comprising:
receiving perception data associated with a host vehicle;
extracting features from the perception data;
detecting a proximate vehicle in a vicinity of the host vehicle using the extracted features;
generating a trajectory of the detected proximate vehicle based on the perception data;
generating a predicted intention of the detected proximate vehicle using the perception data, the trajectory of the detected proximate vehicle, and an intention prediction model;
modifying the predicted intention of the detected proximate vehicle using a Hidden Markov Model (HMM); and
generating a predicted trajectory of the detected proximate vehicle using the modified predicted intention.

17. The non-transitory computer readable storage medium of claim 16 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, comprising:
outputting the modified predicted intention and the predicted trajectory of the detected proximate vehicle to a simulation system.

18. The non-transitory computer readable storage medium of claim 16 , wherein the extracting the features from the perception data is performed using semantic segmentation.

19. The non-transitory computer readable storage medium of claim 16 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, comprising: removing noise and spurious data from the trajectory of the detected proximate vehicle.

20. The non-transitory computer readable storage medium of claim 16 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, comprising:
updating hidden states of the HMM using the predicted trajectory of the detected proximate vehicle.

Assignees

• Tusimple Inc

Inventors

• Yan Zhipeng
• Wang Mingdong
• Liu Siyuan
• Hou Xiaodi

Classifications

• G06V20/584 Primary

  of vehicle lights or traffic lights · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• B60W30/09

  Taking automatic action to avoid collision, e.g. braking and steering · CPC title