Path-conditioned motion forecasting for vehicle motion planning

Patent metadata

Field	Value
Publication number	US-12491868-B2
Application number	US-202117308721-A
Country	US
Kind code	B2
Filing date	May 5, 2021
Priority date	Aug 12, 2020
Publication date	Dec 9, 2025
Grant date	Dec 9, 2025

Abstract

Systems and methods of determining trajectories of an actor in an environment in which a vehicle is operating are provided. The method includes detecting an actor that may move within a scene in the environment by an object detection system of a vehicle in the environment, determining a kinematic history of the actor, and using context of the scene and the kinematic history of the actor to determine a plurality of reference polylines for the actor. The method further includes generating a contextual embedding of the kinematic history of the actor to generate a plurality of predicted trajectories of the actor, in which the generating conditions each of the predicted trajectories to correspond to one of the reference polylines. The method additionally includes using, by the vehicle, the plurality of predicted trajectories to plan movement of the vehicle.

First claim

The invention claimed is:

1 . A method of determining trajectories of an actor in an environment in which a vehicle is operating, the method comprising:
by an object detection system of a vehicle that is moving in an environment, detecting an actor that may move within a scene in the environment; determining a kinematic history of the actor, wherein the kinematic history includes a measured path of movement of the actor; using context of the scene to determine a plurality of reference polylines for the actor, wherein:
determining the plurality of reference polylines comprises constructing the reference polylines from a directed lane graph,
the directed lane graph comprises a plurality of edges that each correspond to one or more lanes and nodes that each correspond to a predicted position of an object within the one or more lanes, and
each reference polyline defines a path along which the actor may travel within the scene over a time period according to the context of the scene;
generating a plurality of predicted trajectories of the actor using the kinematic history and conditioning each of the predicted trajectories to correspond to one of the reference polylines; and by the vehicle,
using the plurality of predicted trajectories to plan movement of the vehicle; and
controlling the vehicle based on the planned movement of the vehicle.

2 . The method of claim 1 , wherein the one or more actions include one or more of the following:
altering a path of the vehicle; altering a pose of the vehicle; or altering a velocity of the vehicle.

3 . The method of claim 1 , wherein using the context of the scene to determine a plurality of reference polylines for the object comprises:
determining a plurality of lane segments along which the actor may travel; and selecting only reference polylines that correspond to lane segments that will not cause the actor to violate a scene context rule.

4 . The method of claim 1 , wherein using the kinematic history of the actor to generate a plurality of predicted trajectories is performed by a decoder module.

5 . The method of claim 1 , further comprising:
predicting a waypoint, wherein the waypoint is a predicted position of the actor at a conclusion of a waypoint time period; identifying a segment of the reference polyline, the segment extending from a current location to a point along the reference polyline that is closest to the waypoint; and determining a first future trajectory state as a trajectory conditioned by the segment of the reference polyline.

6 . The method of claim 1 , wherein using the plurality of predicted trajectories to plan movement of the vehicle comprises:
identifying one or more combinations of locations and times where the first future trajectory state may intersect with a planned trajectory of the vehicle; and causing a motion subsystem of the vehicle to take an action that avoids the identified combinations of locations and times.

7 . A system for determining trajectories of an actor in an environment in which a vehicle is operating, the system comprising:
an object detection system configured to detect an actor that may move within a scene in an environment of a vehicle; and a computer-readable storage medium comprising one or more programming instructions that, when executed, will cause the vehicle to:
determine a kinematic history of the actor, wherein the kinematic history includes a measured path of movement of the actor,
determine a plurality of reference polylines for the actor using context of the scene and the kinematic history, wherein:
determining the plurality of reference polylines comprises constructing the reference polylines from a directed lane graph;
the directed lane graph comprises a plurality of edges that each correspond to one or more lanes and nodes that each correspond to a predicted position of an object within the one or more lanes; and
each reference polyline defines a path along which the actor may travel within the scene over a time period according to the context of the scene,

generate, using a contextual embedding of the kinematic history of the actor, a plurality of predicted trajectories of the actor and condition each of the predicted trajectories to correspond to one of the reference polylines, use the plurality of predicted trajectories to plan one or more movements of the vehicle; and control the vehicle based on the planned one or more movements of the vehicle.

8 . The system of claim 7 , wherein the object detection system includes one or more of the following: a location sensor; one or more cameras; a LiDAR sensor system; a radar system; or a sonar system.

9 . The system of claim 7 , wherein the object detection system includes one or more environmental sensors.

10 . The system of claim 9 , wherein the environmental sensors include one or more of the following: a precipitation sensor; or an ambient temperature sensor.

11 . The system of claim 7 , wherein the instructions to use the context of the scene to determine a plurality of reference polylines for the object comprise-instructions to:
determine a plurality of lane segments along which the actor may travel; and select only reference polylines that correspond to lane segments that will not cause the actor to violate a scene context rule.

12 . The system of claim 7 , further comprising a decoder module configured to use the kinematic history of the actor to generate a plurality of predicted trajectories.

13 . The system of claim 7 , wherein the computer-readable storage medium further includes one or more programming instructions that, when executed, will cause the vehicle to:
predict a waypoint, wherein the waypoint is a predicted position of the actor at a conclusion of a waypoint time period; identify a segment of the reference polyline, the segment extending from a current location to a point along the reference polyline that is closest to the waypoint; and determine a first future trajectory as a trajectory relative to the segment of the reference polyline.

14 . The system of claim 13 , wherein the instructions to use the plurality of predicted trajectories to plan movement of the vehicle comprise instructions to:
identify one or more combinations of locations and times where the first future trajectory may intersect with a planned trajectory of the vehicle; and cause a motion subsystem of the vehicle to take an action that avoids the identified combinations of locations and times.

15 . The system of claim 14 , wherein the one or more actions include one or more of the following:
altering the trajectory of the vehicle; or altering the velocity of the vehicle.

16 . A non-transitory computer-readable storage medium comprising one or more programming instructions that, when executed, will cause a processor to:
use an object detection system of a vehicle to detect an actor that may move within a scene in an environment around the vehicle: determine a kinematic history of the actor, wherein the kinematic history includes a measured path of movement of the actor; determine a plurality of reference polylines for the actor using context of the scene and the kinematic history, wherein:
determining the plurality of reference polylines comprises constructing the reference polylines from a directed lane graph,
the directed lane graph comprises a plurality of edges that each correspond to one or more lanes and nodes that each correspond to a predicted position of an object within the one or more lanes, and
each reference polyline defines a path along which the

Assignees

• Ford Global Tech Llc

Inventors

• Khandelwal Siddhesh Shyam
• Qi William Junbo
• Singh Jagjeet
• Hartnett Andrew T
• Ramanan Deva

Classifications

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• B60W30/0953

  the prediction being responsive to vehicle dynamic parameters · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V40/25

  Recognition of walking or running movements, e.g. gait recognition · CPC title