Systems and methods for automated trajectory prediction

Patent metadata

Field	Value
Publication number	US-11269341-B2
Application number	US-201916456472-A
Country	US
Kind code	B2
Filing date	Jun 28, 2019
Priority date	Jun 28, 2019
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

Systems, methods, and non-transitory computer-readable media can determine a raster representative of a surrounding environment of a vehicle, wherein the raster depicts one or more objects in the surrounding environment of the vehicle. A plurality of trajectory proposals are determined for a first object of the one or more objects. For each trajectory proposal of the plurality of trajectory proposals, a score indicative of a likelihood that the first object will take a trajectory consistent with the trajectory proposal, and an offset for modifying the trajectory proposal are generated. A predicted trajectory is determined for the first object based on the scores and the offsets for the plurality of trajectory proposals.

First claim

What is claimed is:

1. A computer-implemented method comprising:
determining, by a computing system, a raster representative of a surrounding environment of a vehicle, wherein the raster includes visual information that depicts one or more objects in the surrounding environment of the vehicle;
determining, by the computing system, a model based on the visual information included in the raster;
determining, by the computing system, a plurality of trajectory proposals for a first object of the one or more objects using the model;
generating, by the computing system, for each trajectory proposal of the plurality of trajectory proposals;
a respective score indicative of a likelihood that the first object will take a trajectory consistent with the trajectory proposal, and
a respective offset for modifying the trajectory proposal that increases the likelihood that the first object will take the trajectory consistent with the modified trajectory proposal; and

determining, by the computing system, a predicted trajectory for the first object based on the respective scores and the respective offsets for each trajectory proposal of the plurality of trajectory proposals.

2. The computer-implemented method of claim 1 , further comprising:
storing ground truth trajectory information for the first object based on an actual trajectory traveled by the first object, wherein the ground truth trajectory information is used to train a machine learning model.

3. The computer-implemented method of claim 1 , wherein the determining the predicted trajectory for the first object based on the respective scores and the respective offsets for each trajectory proposal comprises:
selecting a first trajectory proposal of the plurality of trajectory proposals based on the respective score for each trajectory proposal, and
modifying the first trajectory proposal with a first offset associated with the first trajectory proposal.

4. The computer-implemented method of claim 1 , wherein each trajectory proposal of the plurality of trajectory proposals is represented by a vector comprising a plurality of position values, each position value is associated with a particular time, and the plurality of position values and the associated times define a potential trajectory.

5. The computer-implemented method of claim 4 , wherein each offset comprises a plurality of offset values for modifying the plurality of position values in the trajectory proposal associated with the offset.

6. The computer-implemented method of claim 5 , wherein each trajectory proposal of the plurality of trajectory proposals can be converted into a modified trajectory proposal by combining the trajectory proposal with its associated offset, and each modified trajectory proposal would result in a higher score output from a machine learning model than its associated trajectory proposal.

7. The computer-implemented method of claim 1 , wherein the raster is a two-dimensional image.

8. The computer-implemented method of claim 1 , wherein the raster further comprises semantic map information.

9. The computer-implemented method of claim 1 , wherein the raster further comprises previous trajectory information for each object of the one or more objects.

10. The computer-implemented method of claim 9 , wherein at least some of the plurality of trajectory proposals are automatically generated based on previous trajectory information for the first object.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
determining a raster representative of a surrounding environment of a vehicle, wherein the raster includes visual information that depicts one or more objects in the surrounding environment of the vehicle;
determining a model based on the visual information included in the raster;
determining a plurality of trajectory proposals for a first object of the one or more objects using the model;
generating, for each trajectory proposal of the plurality of trajectory proposals;
a respective score indicative of a likelihood that the first object will take a trajectory consistent with the trajectory proposal, and
a respective offset for modifying the trajectory proposal that increases the likelihood that the first object will take the trajectory consistent with the modified trajectory proposal; and

determining a predicted trajectory for the first object based on the respective scores and the respective offsets for each trajectory proposal of the plurality of trajectory proposals.

12. The system of claim 11 , wherein the instructions, when executed by the at least one processor, further cause the system to perform:
storing ground truth trajectory information for the first object based on an actual trajectory traveled by the first object, wherein the ground truth trajectory information is used to train a machine learning model.

13. The system of claim 11 , wherein the determining the predicted trajectory for the first object based on the respective scores and the respective offsets for each trajectory proposal comprises:
selecting a first trajectory proposal of the plurality of trajectory proposals based on the respective score for each trajectory proposal, and
modifying the first trajectory proposal with a first offset associated with the first trajectory proposal.

14. The system of claim 11 , wherein each trajectory proposal of the plurality of trajectory proposals is represented by a vector comprising a plurality of position values, each position value is associated with a particular time, and the plurality of position values and the associated times define a potential trajectory.

15. The system of claim 14 , wherein each offset comprises a plurality of offset values for modifying the plurality of position values in the trajectory proposal associated with the offset.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
determining a raster representative of a surrounding environment of a vehicle, wherein the raster includes visual information that depicts one or more objects in the surrounding environment of the vehicle;
determining a model based on the visual information included in the raster;
determining a plurality of trajectory proposals for a first object of the one or more objects using the model;
generating, for each trajectory proposal of the plurality of trajectory proposals;
a respective score indicative of a likelihood that the first object will take a trajectory consistent with the trajectory proposal, and
a respective offset for modifying the trajectory proposal that increases the likelihood that the first object will take the trajectory consistent with the modified trajectory proposal; and

determining a predicted trajectory for the first object based on the respective scores and the respective offsets for each trajectory proposal of the plurality of trajectory proposals.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions, when executed by the at least one processor of the computing system, further cause the computing system to perform:
storing ground truth trajectory information for the first object based on an actual trajectory traveled by the first object, wherein the ground truth trajectory information is used to train a machine learning model.

18. The non-transitory computer-readable

Assignees

• Woven Planet North America Inc

Inventors

• Benisch Michael Jared
• Houston John

Classifications

• B60W60/0011 Primary

  involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles · CPC title

• G05D2201/0213

  Physics · mapped topic

• G05D1/0214

  in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13) · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G05D1/0221 Primary

  involving a learning process · CPC title