Agent prioritization for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11048253-B2
Application number	US-201816198130-A
Country	US
Kind code	B2
Filing date	Nov 21, 2018
Priority date	Nov 21, 2018
Publication date	Jun 29, 2021
Grant date	Jun 29, 2021

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying high-priority agents in the vicinity of a vehicle and, for only those agents which are high priority agents, generating data characterizing the agents using a first prediction model. In a first aspect, a system identifies multiple agents in an environment in a vicinity of a vehicle. The system generates a respective importance score for each of the agents by processing a feature representation of each agent using an importance scoring model. The importance score for an agent characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle. The system identifies, as high-priority agents, a proper subset of the plurality of agents with the highest importance scores.

First claim

What is claimed is:

1. A method performed by one or more data processing apparatus, the method comprising:
identifying a plurality of agents currently located at respective locations in an environment in a vicinity of a vehicle;
generating a respective importance score for each of the plurality of agents, comprising, for each agent:
processing a feature representation of the agent using an importance scoring model to generate an importance score for the agent,
wherein the importance score for the agent characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle;

selecting, for one or more of the agents, a respective prediction model for use in generating data characterizing the agent based on the importance score for the agent, comprising:
identifying, as high-priority agents, a proper subset of the plurality of agents with the highest importance scores; and
selecting, for only those agents of the plurality of agents that are identified as high-priority agents, a first prediction model for use in generating data characterizing the agent; and

generating, for each of the high-priority agents, data characterizing the agent using the first prediction model selected for the agent; and
providing the data characterizing the high-priority agents generated using the first prediction model to the planning system of the vehicle to generate the planning decisions which plan the future trajectory of the vehicle.

2. The method of claim 1 , wherein identifying a plurality of agents currently located at respective locations in an environment in a vicinity of a vehicle comprises:
identifying the plurality of agents in the environment in the vicinity of the vehicle using sensor data captured by one or more sensors of the vehicle.

3. The method of claim 1 , further comprising generating a respective feature representation of each of the plurality of agents using sensor data captured by one or more sensors of the vehicle, wherein the feature representation of an agent comprises one or more of: a velocity of the agent, an acceleration of the agent, a type of the agent, a distance from the agent to the vehicle, and data indicating whether a predicted trajectory of the agent will overlap a trajectory of the vehicle.

4. The method of claim 1 , wherein identifying, as high-priority agents, a proper subset of the plurality of agents with the highest importance scores comprises:
identifying, as high-priority agents, a predetermined number of the plurality of agents with the highest importance scores; or
identifying, as high-priority agents, each agent of the plurality of agents with an importance score that satisfies a predetermined threshold.

5. The method of claim 1 , further comprising:
providing the planning decisions which plan the future trajectory of the vehicle to a control system of the vehicle for use in controlling operation of the vehicle.

6. The method of claim 1 , wherein each operation of the method is performed on-board the vehicle.

7. The method of claim 1 , wherein selecting, for one or more of the agents, a respective prediction model for use in generating data characterizing the agent based on the importance score for the agent further comprises:
identifying, as low-priority agents, one or more of the agents that are not identified as high-priority agents; and
selecting, for each agent identified as a low-priority agent, a second prediction model for use in generating data characterizing the agent, wherein the second prediction model is different than the first prediction model;
wherein the method further comprises:
generating, for each of the low-priority agents, data characterizing the agent using the second prediction model;
providing, to the planning system of the vehicle, the data characterizing the low-priority agents generated using the second prediction model in addition to the data characterizing the high-priority agents generated using the first prediction model.

8. The method of claim 7 , wherein the first prediction model has a first number of model parameters, the second prediction model has a second number of model parameters, and the second number of model parameters is less than the first number of model parameters.

9. The method of claim 7 , wherein the first prediction model and the second prediction model are implemented as respective neural network systems.

10. The method of claim 1 , wherein the first prediction model is a first behavior prediction model; and
wherein generating, for each of the high-priority agents, data characterizing the agent using the first prediction model selected for the agent comprises:
generating, for each of the high-priority agents, data characterizing predicted future behavior of the agent using the first behavior prediction model.

11. The method of claim 10 , wherein generating data characterizing the predicted future behavior of the agent comprises generating, for each of multiple spatial locations in the environment, a respective probability that the agent will occupy the spatial location at a specified time point after a current time point.

12. A system, comprising:
a data processing apparatus;
a memory in data communication with the data processing apparatus and storing instructions that cause the data processing apparatus to perform operations comprising:
identifying a plurality of agents currently located at respective locations in an environment in a vicinity of a vehicle;
generating a respective importance score for each of the plurality of agents, comprising, for each agent:
processing a feature representation of the agent using an importance scoring model to generate an importance score for the agent,
wherein the importance score for the agent characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle;

selecting, for one or more of the agents, a respective prediction model for use in generating data characterizing the agent based on the importance score for the agent, comprising:
identifying, as high-priority agents, a proper subset of the plurality of agents with the highest importance scores; and
selecting, for only those agents of the plurality of agents that are identified as high-priority agents, a first prediction model for use in generating data characterizing the agent; and

generating, for each of the high-priority agents, data characterizing the agent using the first prediction model selected for the agent; and
providing the data characterizing the high-priority agents generated using the first prediction model to the planning system of the vehicle to generate the planning decisions which plan the future trajectory of the vehicle.

13. The system of claim 12 , wherein identifying a plurality of agents currently located at respective locations in an environment in a vicinity of a vehicle comprises:
identifying the plurality of agents in the environment in the vicinity of the vehicle using sensor data captured by one or more sensors of the vehicle.

14. The system of claim 12 , wherein the operations further comprise generating a respective feature representation of each of the plurality of agents using sensor data captured by one or more sensors of the vehicle, wherein the feature representation of an agent comprises one or more of: a velocity of the agent, an acceleration of the agent, a type of the agent, a distance from the agent to the vehicle, and data indicating whether a predicted trajectory of the agent will overlap a traje

Assignees

• Waymo Llc

Inventors

• Ding Kai
• Refaat Khaled
• Ross Stephane

Classifications

• B60W60/00276 Primary

  for two or more other traffic participants · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title