System and method for proximate vehicle intention prediction for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11104334-B2
Application number	US-201815994138-A
Country	US
Kind code	B2
Filing date	May 31, 2018
Priority date	May 31, 2018
Publication date	Aug 31, 2021
Grant date	Aug 31, 2021

Abstract

A system and method for proximate vehicle intention prediction for autonomous vehicles are disclosed. A particular embodiment is configured to: receive perception data associated with a host vehicle; extract features from the perception data to detect a proximate vehicle in the vicinity of the host vehicle; generate a trajectory of the detected proximate vehicle based on the perception data; use a trained intention prediction model to generate a predicted intention of the detected proximate vehicle based on the perception data and the trajectory of the detected proximate vehicle; use the predicted intention of the detected proximate vehicle to generate a predicted trajectory of the detected proximate vehicle; and output the predicted intention and predicted trajectory for the detected proximate vehicle to another subsystem.

First claim

What is claimed is:

1. A system comprising:
a data processor; and
a memory for storing a proximate vehicle intention prediction module, executable by the data processor, the proximate vehicle intention prediction module being configured to perform a proximate vehicle intention prediction operation for autonomous vehicles, the proximate vehicle intention prediction operation being configured to:
receive perception data associated with a host vehicle;
extract features from the perception data to detect a proximate vehicle in a vicinity of the host vehicle;
generate a trajectory of the detected proximate vehicle based on the perception data;
produce a smoothed trajectory of the detected proximate vehicle by performing a temporal smoothing of the trajectory of the detected proximate vehicle;
generate a predicted intention of the detected proximate vehicle using the perception data, the smoothed trajectory of the detected proximate vehicle, and a trained intention prediction model;
generate a predicted trajectory of the detected proximate vehicle using the predicted intention of the detected proximate vehicle; and
output the predicted intention and the predicted trajectory for the detected proximate vehicle to another subsystem,
wherein said generate the predicted intention of the detected proximate vehicle comprises:
generate a distribution of probabilistic maneuvers associated with the detected proximate vehicle using the perception data and the smoothed trajectory of the detected proximate vehicle;
obtain a filtered distribution of probabilistic maneuvers associated with the detected proximate vehicle by applying a Bayesian filter to the distribution of probabilistic maneuvers associated with the detected proximate vehicle; and
modify the filtered distribution of probabilistic maneuvers associated with the detected proximate vehicle using a Hidden Markov Model (HMM).

2. The system of claim 1 wherein the perception data comprises data received from at least one of: a camera, an image capture device, an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a RADAR unit, or a laser range finder/LIDAR unit.

3. The system of claim 1 being further configured to use semantic segmentation to extract features from the perception data.

4. The system of claim 1 being further configured to train the intention prediction model with training data gathered during an offline training phase.

5. The system of claim 1 being further configured to generate the trajectory of the detected proximate vehicle by aggregating perception data for the detected proximate vehicle across multiple image frames using object tracking identifiers.

6. The system of claim 1 being further configured to filter the trajectory of the detected proximate vehicle.

7. The system of claim 1 wherein the predicted intention and predicted trajectory for the detected proximate vehicle are output to a vehicle system causing the host vehicle to follow a proposed motion plan.

8. A method comprising:
receiving perception data associated with a host vehicle;
extracting features from the perception data to detect a proximate vehicle in a vicinity of the host vehicle;
generating a trajectory of the detected proximate vehicle based on the perception data;
producing a smoothed trajectory of the detected proximate vehicle by performing a temporal smoothing of the trajectory of the detected proximate vehicle;
generating a predicted intention of the detected proximate vehicle using the perception data, the smoothed trajectory of the detected proximate vehicle, and a trained intention prediction model;
generating a predicted trajectory of the detected proximate vehicle using the predicted intention of the detected proximate vehicle; and
output the predicted intention and the predicted trajectory for the detected proximate vehicle to another subsystem,
wherein said generating the predicted intention of the detected proximate vehicle comprises:
generating a distribution of probabilistic maneuvers associated with the detected proximate vehicle using the perception data and the smoothed trajectory of the detected proximate vehicle;
obtaining a filtered distribution of probabilistic maneuvers associated with the detected proximate vehicle by applying a Bayesian filter to the distribution of probabilistic maneuvers associated with the detected proximate vehicle; and
modifying the filtered distribution of probabilistic maneuvers associated with the detected proximate vehicle using a Hidden Markov Model (HMM).

9. The method of claim 8 wherein the perception data comprises data received from a sound navigation and ranging (sonar) device.

10. The method of claim 8 comprising using semantic segmentation on image frame sequences from the perception data to identify the proximate vehicle.

11. The method of claim 8 comprising configuring parameters used for the trained intention prediction model during an offline training phase.

12. The method of claim 8 comprising removing or filtering outlier data corresponding to the trajectory of the detected proximate vehicle.

13. The method of claim 8 comprising removing noise and spurious data of the trajectory of the detected proximate vehicle.

14. The method of claim 8 wherein the host vehicle follows an output proposed motion plan corresponding to the predicted intention and the predicted trajectory for the detected proximate vehicle.

15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive perception data associated with a host vehicle;
extract features from the perception data to detect a proximate vehicle in a vicinity of the host vehicle;
generate a trajectory of the detected proximate vehicle based on the perception data;
produce a smoothed trajectory of the detected proximate vehicle by performing a temporal smoothing of the trajectory of the detected proximate vehicle;
generate a predicted intention of the detected proximate vehicle using the perception data, the smoothed trajectory of the detected proximate vehicle, and a trained intention prediction model;
generate a predicted trajectory of the detected proximate vehicle using the predicted intention of the detected proximate vehicle; and
output the predicted intention and the predicted trajectory for the detected proximate vehicle to another subsystem,
wherein said generate the predicted intention of the detected proximate vehicle comprises:
generate a distribution of probabilistic maneuvers associated with the detected proximate vehicle using the perception data and the smoothed trajectory of the detected proximate vehicle;
obtain a filtered distribution of probabilistic maneuvers associated with the detected proximate vehicle by applying a Bayesian filter to the distribution of probabilistic maneuvers associated with the detected proximate vehicle; and
modify the filtered distribution of probabilistic maneuvers associated with the detected proximate vehicle using a Hidden Markov Model (HMM).

16. The non-transitory machine-usable storage medium of claim 15 , wherein the perception data comprises data received from at least one of: a camera, an image capture device, an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a RADAR unit, or a laser range finder/LIDAR unit.

17. The non-transitory machine-usable storage medium of claim 15 , wherein the instructions, when executed by the machine, cause the machine to use semantic segmentation to extract features from t

Assignees

• Tusimple Inc

Inventors

• Yan Zhipeng
• Wang Mingdong
• Liu Siyuan
• Hou Xiaodi

Classifications

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• B60W30/0956 Primary

  the prediction being responsive to traffic or environmental parameters · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F18/295

  Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models · CPC title