System and method for vehicle lane change prediction using structural recurrent neural networks

Patent metadata

Field	Value
Publication number	US-2019077398-A1
Application number	US-201715858592-A
Country	US
Kind code	A1
Filing date	Dec 29, 2017
Priority date	Sep 14, 2017
Publication date	Mar 14, 2019
Grant date	—

Abstract

System, methods, and other embodiments described herein relate to predicting lane changes for nearby vehicles of a host vehicle. In one embodiment, a method includes, in response to detecting that one or more of the nearby vehicles are present proximate to the host vehicle, collecting pose information about the nearby vehicles. The nearby vehicles are traveling proximate to the host vehicle and in a direction of the host vehicle. The method includes analyzing the pose information of the nearby vehicles using separate recurrent units of a structural recurrent neural network (S-RNN) to generate factors according to the lanes. The method includes generating prediction indicators for the nearby vehicles as a function of the factors for the lanes using the S-RNN. The method includes providing electronic outputs identifying the prediction indicators that specify a likelihood of the nearby vehicles changing between the lanes.

First claim

What is claimed is:

1 . A lane prediction system for predicting lane changes for nearby vehicles of a host vehicle, comprising:
one or more processors; a memory communicably coupled to the one or more processors and storing: a monitoring module including instructions that when executed by the one or more processors cause the one or more processors to, in response to detecting that one or more of the nearby vehicles are present proximate to the host vehicle, collect, using at least one sensor of the host vehicle, pose information about the nearby vehicles, wherein the one or more of the nearby vehicles are traveling in lanes of a roadway proximate to the host vehicle and in a direction of travel of the host vehicle; and a prediction module including instructions that when executed by the one or more processors cause the one or more processors to:
analyze the pose information of the nearby vehicles using separate recurrent units of a structural recurrent neural network (S-RNN) to generate factors according to the lanes,
generate prediction indicators for the nearby vehicles as a function of the factors for the lanes using the S-RNN, and
provide electronic outputs identifying the prediction indicators that specify a likelihood of the nearby vehicles changing between the lanes.

2 . The lane prediction system of claim 1 , wherein the recurrent units of the S-RNN are modeled from a factor graph for the lanes and positions of respective ones of the nearby vehicles, wherein the recurrent units are representative of factors of the respective lanes and spatiotemporal interactions of respective ones of the nearby vehicles in the respective lanes, and wherein the recurrent units account for the lanes including three lanes about the host vehicle and for each of the three lanes including two representative vehicles.

3 . The lane prediction system of claim 2 , wherein the recurrent units are long short-term memory (LSTM) recurrent units.

4 . The lane prediction system of claim 1 , wherein the prediction module includes instructions to generate the prediction indicators including instructions to iteratively generating the prediction indicators at successive time steps to maintain the prediction indicators to a prediction horizon that is a defined period of time into the future for which the prediction indicators are forecast,
wherein the pose information includes at least a position, orientation, and trajectory of the one or more of the nearby vehicles relative to the host vehicle, and wherein the recurrent units are associated with a left lane directly to a left of the host vehicle, a same lane as the host vehicle, and a right lane directly to a right of the host vehicle.

5 . The lane prediction system of claim 1 , wherein the monitoring module includes instructions to detect that the one or more of the nearby vehicles are present includes instructions to identifying that the one or more of the nearby vehicles are within a defined distance of the host vehicle by monitoring sensor data for indicators of a presence of the nearby vehicles,
wherein each of the nearby vehicles is associated with a position within a factor graph that associates the nearby vehicles with one of the lanes and a relative position in relation to the host vehicle, and wherein the monitoring module includes instructions to detect the one or more of the nearby vehicles including instructions to analyze map data for the roadway on which the host vehicle is traveling to determine a number of the lanes for a direction of travel of the host vehicle.

6 . The lane prediction system of claim 1 , wherein the prediction module includes instructions to provide the electronic outputs including instructions to provide the electronic outputs to at least an autonomous driving module to cause the autonomous driving module to execute path planning for controlling the host vehicle according to the electronic outputs.

7 . The lane prediction system of claim 1 , further comprising:
training, using stored sensor data, the S-RNN to learn separate factor functions of a factor graph for the lanes, wherein the recurrent units are structured to correlate with the factor graph, and wherein generating the prediction indicators includes concatenating the factors and analyzing the concatenated factors with a node RNN of the S-RNN, and wherein training the S-RNN includes learning weights and biases for fully connected layers in the S-RNN.

8 . The lane prediction system of claim 1 , wherein the host vehicle is an autonomous vehicle.

9 . A non-transitory computer-readable medium storing for predicting lane changes for nearby vehicles of a host vehicle and including instructions that when executed by one or more processors cause the one or more processors to:
in response to detecting that one or more of the nearby vehicles are present proximate to the host vehicle, collect, using at least one sensor of the host vehicle, pose information about the nearby vehicles, wherein the one or more of the nearby vehicles are traveling in lanes of a roadway proximate to the host vehicle and in a direction of travel of the host vehicle; analyze the pose information of the nearby vehicles using separate recurrent units of a structural recurrent neural network (S-RNN) to generate factors according to the lanes; generate prediction indicators for the nearby vehicles as a function of the factors for the lanes using the S-RNN; and provide electronic outputs identifying the prediction indicators that specify a likelihood of the nearby vehicles changing between the lanes.

10 . The non-transitory computer-readable medium of claim 9 , wherein the recurrent units of the S-RNN are modeled from a factor graph for the lanes and positions of respective ones of the nearby vehicles, wherein the recurrent units are representative of factors of the respective lanes and spatiotemporal interactions of respective ones of the nearby vehicles in the respective lanes, and
wherein the recurrent units account for the lanes including three lanes about the host vehicle and for each of the three lanes including two representative vehicles.

11 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to generate the prediction indicators include instructions to iteratively generating the prediction indicators at successive time steps to maintain the prediction indicators to a prediction horizon that is a defined period of time into the future for which the prediction indicators are forecast,
wherein the pose information includes at least a position, orientation, and trajectory of the one or more of the nearby vehicles relative to the host vehicle, and wherein the recurrent units are associated with a left lane directly to a left of the host vehicle, a same lane as the host vehicle, and a right lane directly to a right of the host vehicle.

12 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to detect that the one or more of the nearby vehicles are present includes instructions to identify that the one or more of the nearby vehicles are within a defined distance of the host vehicle by monitoring sensor data for indicators of a presence of the nearby vehicles,
wherein each of the nearby vehicles is associated with a position within a factor graph that associates the nearby vehicles with one of the lanes and a relative position in relation to the host vehicle, and wherein the instructions to detecting the one or more of the nearby vehicles include instructions to analyze map data for the roadway on which the host vehicle is traveling to determine a number of the lanes for a direction of travel of the h

Assignees

• Toyota Eng & Mfg North America
• Univ Michigan Regents

Inventors

• Kusano Kristofer D
• Patel Sajan
• Griffin Brent
• Corso Jason J

Classifications

• B60W60/00276

  for two or more other traffic participants · CPC title

• B60W60/001

  Planning or execution of driving tasks · CPC title

• B60W60/00274

  considering possible movement changes · CPC title

• B60W2554/4045

  Intention, e.g. lane change or imminent movement · CPC title

• G06V10/82 Primary

  using neural networks · CPC title