Systems and methods for selecting trajectories based on interpretable semantic representations

Patent metadata

Field	Value
Publication number	US-11834069-B2
Application number	US-202117150984-A
Country	US
Kind code	B2
Filing date	Jan 15, 2021
Priority date	Mar 5, 2020
Publication date	Dec 5, 2023
Grant date	Dec 5, 2023

Abstract

Systems and methods for generating semantic occupancy maps are provided. In particular, a computing system can obtain map data for a geographic area and sensor data obtained by the autonomous vehicle. The computer system can identify feature data included in the map data and sensor data. The computer system can, for a respective semantic object type from a plurality of semantic object types, determine, by the computing system and using feature data as input to a respective machine-learned model from a plurality of machine-learned models, one or more occupancy maps for one or more timesteps in the future, and wherein the respective machine-learned model is trained to determine occupancy for the respective semantic object type. The computer system can select a trajectory for the autonomous vehicle based on a plurality of occupancy maps associated with the plurality of semantic object types.

First claim

What is claimed is:

1. A computer-implemented method for autonomous vehicle motion control, the method comprising:
obtaining, by a computing system including one or more processors, map data for a geographic area and sensor data obtained by the autonomous vehicle;
identifying, by the computing system, feature data included in the map data and sensor data;
determining, by the computing system and using feature data as input to a plurality of machine-learned models, a plurality of sets of occupancy maps, each set of occupancy maps corresponding to a unique semantic object type for one or more timesteps in the future, and wherein the respective machine-learned model is trained to determine occupancy for each of the plurality of semantic object types; and
selecting, by the computing system, a trajectory for the autonomous vehicle based on a plurality of occupancy maps associated with the plurality of semantic object types.

2. The computer-implemented method of claim 1 , wherein semantic object types are organized into a semantic hierarchy.

3. The computer-implemented method of claim 1 , wherein the semantic object types can include pedestrians, bicycles, and vehicles.

4. The computer-implemented method of claim 3 , wherein each semantic object type includes one or more subtypes.

5. The computer-implemented method of claim 4 , wherein the one or more subtypes can include occluded object types.

6. The computer-implemented method of claim 1 , further comprising:
generating, by the computing system, a fused representation of the feature data included in the map data and the sensor data.

7. The computer-implemented method of claim 1 , wherein a particular occupancy map includes a grid of points, each point representing a specific area of the environment around the autonomous vehicle at a particular time step.

8. The computer-implemented method of claim 7 , wherein each point in the grid of points includes occupancy data representing whether the represented area includes an object of the semantic object type associated with the occupancy map.

9. The computer-implemented method of claim 7 , wherein each point in the grid of points includes a confidence value.

10. The computer-implemented method of claim 1 , wherein selecting, by the computing system, a trajectory for the autonomous vehicle based on a plurality of occupancy maps associated with the plurality of semantic object types further comprises:
generating, by the computing system, a plurality of candidate trajectories.

11. The computer-implemented method of claim 10 , wherein selecting, by the computing system, a trajectory for the autonomous vehicle based on a plurality of occupancy maps associated with the plurality of semantic object types further comprises:
for a respective candidate trajectory in the plurality of candidate trajectories:
determining, by the computing system, a cost associated with the respective candidate trajectory; and
selecting, by the computing system, a trajectory from the plurality of trajectories based, at least in part, on the costs associated with the plurality of trajectories.

12. The computer-implemented method of claim 11 , wherein the costs associated with the respective candidate trajectory is based, at least in part, on the plurality of occupancy maps.

13. The computer-implemented method of claim 1 , wherein the map data includes data describing the position of lanes, boundaries, crossing areas, and traffic control mechanisms.

14. The computer-implemented method of claim 1 , wherein the sensor data is generated by a LIDAR sensor and comprises multiple LIDAR sweeps.

15. The computer-implemented method of claim 1 , wherein each machine-learned model in the plurality of machine-learned models is associated with a specific semantic object type.

16. The computer-implemented method of claim 1 , wherein the plurality of machine-learned models are recurrent neural networks.

17. A computing system for an autonomous vehicle, the system comprising:
one or more processors and one or more non-transitory computer-readable memories;
wherein the one or more non-transitory computer-readable memories store instructions that, when executed by the processor, cause the computing system to perform operations, the operations comprising:
obtaining map data for a geographic area and sensor data obtained by the autonomous vehicle;
identifying, using one or more machine-learned models, feature data included in the map data and sensor data;
determining, using feature data as input to a plurality of machine-learned models, a plurality of sets of occupancy maps, each set of occupancy maps corresponding to a unique semantic object type for one or more timesteps in the future; and

selecting a trajectory for the autonomous vehicle based on a plurality of occupancy maps associated with the plurality of semantic object types.

18. The computing system of claim 17 , wherein semantic object types are organized into a semantic hierarchy.

19. The computing system of claim 17 , wherein the semantic object types can include pedestrians, bicycles, and vehicles.

20. An autonomous vehicle, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining map data for a geographic area and sensor data obtained by the autonomous vehicle;
identifying, using one or more machine-learned models, feature data included in the map data and sensor data;
determining using feature data as input to a plurality of machine-learned models, a plurality of sets of occupancy maps, each set of occupancy maps corresponding to a unique semantic object type for one or more timesteps in the future; and
selecting a trajectory for the autonomous vehicle based on a plurality of occupancy maps associated with the plurality of semantic object types.

Assignees

• Uatc Llc
• Uatc Lcc

Inventors

• Urtasun Raquel
• Sadat Abbas
• Casas Sergio
• Ren Mengye

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• B60W60/0011 Primary

  involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles · CPC title

• G06F18/213

  Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods · CPC title