Agent prioritization for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11673550-B2
Application number	US-202117320727-A
Country	US
Kind code	B2
Filing date	May 14, 2021
Priority date	Nov 20, 2018
Publication date	Jun 13, 2023
Grant date	Jun 13, 2023

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying high-priority agents in the vicinity of a vehicle. In one aspect, a method comprises processing an input that characterizes a trajectory of the vehicle in an environment using an importance scoring model to generate an output that defines a respective importance score for each of a plurality of agents in the environment in the vicinity of the vehicle. The importance score for an agent characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle. The high-priority agents are identified as a proper subset of the plurality of agents with the highest importance scores.

First claim

What is claimed is:

1. A method performed by one or more data processing apparatus, the method comprising:
generating, using an importance scoring model and for each of a plurality of agents currently located at respective locations in an environment in a vicinity of a vehicle, a respective importance score for the agent that characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle;
selecting, for one or more of the agents, a respective prediction model for use in generating data characterizing the agent based on the importance score for the agent, comprising:
identifying a proper subset of the plurality of agents as high-priority agents based on their respective importance scores, comprising:
identifying one or more agents of the plurality of agents as having highest importance scores from among the plurality of agents;
designating the one or more agents identified as having the highest importance scores from among the plurality of agents as being high-priority agents;

selecting, for only those agents of the plurality of agents that are identified as high-priority agents based on their respective importance scores, a first prediction model for use in generating data characterizing the agent; and

generating, for each of the high-priority agents, data characterizing the agent using the first prediction model selected for the agent; and
providing the data characterizing the high-priority agents generated using the first prediction model to the planning system of the vehicle to generate the planning decisions which plan the future trajectory of the vehicle.

2. The method of claim 1 , wherein generating, using the importance scoring model and for each of the plurality of agents currently located at respective locations in the environment in the vicinity of the vehicle, the respective importance score for the agent, comprises:
processing an input that characterizes a trajectory of the vehicle in the environment using the importance scoring model to generate an output that defines the respective importance score for each agent.

3. The method of claim 2 , wherein:
the output of the importance scoring model comprises an output channel that is represented as a two-dimensional array of data values;
each position in the output channel corresponds to a respective spatial position in the environment; and
for each spatial position in the environment that is occupied by an agent of the plurality of agents at a current time point, the position in the output channel that corresponds to the spatial position defines an importance score of the agent.

4. The method of claim 3 , further comprising, for each agent of the plurality of agents:
generating a respective feature representation of the agent, comprising:
generating one or more importance score features of the agent from the output channel;
generating one or more additional features of the agent based on sensor data captured by one or more sensors of the vehicle; and
generating the feature representation of the agent from the importance score features of the agent and the additional features of the agent;

processing the feature representation of the agent using an importance score refining model to generate a refined importance score for the agent that characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle.

5. The method of claim 2 , further comprising:
obtaining historical data characterizing the trajectory of the vehicle in the environment, the historical data comprising, for each of a plurality of previous time points, data defining: (i) a spatial position in the environment occupied by the vehicle at the previous time point, and (ii) respective values of each motion parameter in a predetermined set of motion parameters, wherein the value of each motion parameter characterizes a respective feature of a motion of the vehicle at the previous time point;
generating a representation of the trajectory of the vehicle in the environment, wherein:
the representation of the trajectory of the vehicle in the environment is a concatenation of a plurality of channels;
each channel is represented as a two-dimensional array of data values;
each position in each channel corresponds to a respective spatial position in the environment;
corresponding positions in different channels correspond to the same spatial position in the environment;
the channels comprise a time channel and a respective motion channel corresponding to each motion parameter in the predetermined set of motion parameters; and
for each particular spatial position in the environment occupied by the vehicle at a particular previous time point:
the position in the time channel which corresponds to the particular spatial position defines the particular previous time point; and
for each motion channel, the position in the motion channel which corresponds to the particular spatial position defines the value of the motion parameter corresponding to the motion channel at the particular previous time point;

wherein processing an input that characterizes a trajectory of the vehicle in the environment comprises processing an input that includes the representation of the trajectory of the vehicle in the environment.

6. The method of claim 5 , wherein obtaining the respective values of each motion parameter in the predetermined set of motion parameters for a previous time point comprises one or more of:
obtaining the value of a velocity motion parameter which characterizes a velocity of the vehicle at the previous time point;
obtaining the value of an acceleration motion parameter which characterizes an acceleration of the vehicle at the previous time point; and
obtaining the value of a heading motion parameter which characterizes a heading of the vehicle at the previous time point.

7. The method of claim 2 , wherein the input processed by the importance scoring model comprises one or more of: (i) a road-graph channel representing a known geometry of the environment, (ii) a vehicle localization channel which represents a spatial position of the vehicle in the environment at a current time point by a vehicle bounding box, and (iii) an auxiliary localization channel which represents respective spatial positions of the plurality of agents in the environment at a current time point by respective bounding boxes.

8. The method of claim 2 , further comprising generating a joint representation of trajectories of the plurality of agents in the environment in the vicinity of the vehicle, wherein the input processed by the importance scoring model further comprises the joint representation of the trajectories of the plurality of agents.

9. The method of claim 8 , wherein:
the joint representation of the trajectories of the plurality of agents in the environment comprises an auxiliary time channel and a respective auxiliary motion channel corresponding to each motion parameter in a predetermined set of motion parameters;
each channel is represented as a two-dimensional array of data values and each data value in each channel corresponds to a respective spatial position in the environment; and
for each particular spatial position in the environment occupied by a particular agent of the plurality of agents at a particular previous time point:
the data value in the auxiliary time channel which corresponds to the particular spatial position defines the particular previous time point; and
for each auxiliary motion channel, the data value in the auxiliary motion channel which corresponds to the particular spatial position d

Assignees

• Waymo Llc

Inventors

• Ding Kai
• Refaat Khaled
• Ross Stephane

Classifications

• B60W30/0956 Primary

  the prediction being responsive to traffic or environmental parameters · CPC title

• B60W2050/0075

  Automatic parameter input, automatic initialising or calibrating means · CPC title

• B60W2556/10

  Historical data · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N20/00

  Machine learning · CPC title