Agent prioritization for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11034348-B2
Application number	US-201916264136-A
Country	US
Kind code	B2
Filing date	Jan 31, 2019
Priority date	Nov 20, 2018
Publication date	Jun 15, 2021
Grant date	Jun 15, 2021

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying high-priority agents in the vicinity of a vehicle. In one aspect, a method comprises processing an input that characterizes a trajectory of the vehicle in an environment using an importance scoring model to generate an output that defines a respective importance score for each of a plurality of agents in the environment in the vicinity of the vehicle. The importance score for an agent characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle. The high-priority agents are identified as a proper subset of the plurality of agents with the highest importance scores.

First claim

What is claimed is:

1. A method performed by one or more data processing apparatus, the method comprising:
processing an input that characterizes a trajectory of a vehicle in an environment using an importance scoring model to generate an output that defines a respective importance score for each of a plurality of agents currently located at respective locations in the environment in a vicinity of the vehicle,
wherein the importance score for an agent characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle;
selecting, for one or more of the agents, a respective prediction model for use in generating data characterizing the agent based on the importance score for the agent, comprising:
identifying, as high-priority agents, a proper subset of the plurality of agents with the highest importance scores; and
selecting, for only those agents of the plurality of agents that are identified as high-priority agents, a first prediction model for use in generating data characterizing the agent; and

generating, for each of the high-priority agents, data characterizing the agent using the first prediction model selected for the agent; and
providing the data characterizing the high-priority agents generated using the first prediction model to the planning system of the vehicle to generate the planning decisions which plan the future trajectory of the vehicle.

2. The method of claim 1 , further comprising:
obtaining historical data characterizing the trajectory of the vehicle in the environment, the historical data comprising, for each of a plurality of previous time points, data defining: (i) a spatial position in the environment occupied by the vehicle at the previous time point, and (ii) respective values of each motion parameter in a predetermined set of motion parameters, wherein the value of each motion parameter characterizes a respective feature of a motion of the vehicle at the previous time point;
generating a representation of the trajectory of the vehicle in the environment, wherein:
the representation of the trajectory of the vehicle in the environment is a concatenation of a plurality of channels;
each channel is represented as a two-dimensional array of data values;
each position in each channel corresponds to a respective spatial position in the environment;
corresponding positions in different channels correspond to the same spatial position in the environment;
the channels comprise a time channel and a respective motion channel corresponding to each motion parameter in the predetermined set of motion parameters; and
for each particular spatial position in the environment occupied by the vehicle at a particular previous time point:
the position in the time channel which corresponds to the particular spatial position defines the particular previous time point; and
for each motion channel, the position in the motion channel which corresponds to the particular spatial position defines the value of the motion parameter corresponding to the motion channel at the particular previous time point;

wherein processing an input that characterizes a trajectory of the vehicle in the environment comprises processing an input that includes the representation of the trajectory of the vehicle in the environment.

3. The method of claim 2 , wherein:
the output of the importance scoring model comprises an output channel that is represented as a two-dimensional array of data values;
each position in the output channel corresponds to a respective spatial position in the environment; and
for each spatial position in the environment that is occupied by an agent of the plurality of agents at a current time point, the position in the output channel that corresponds to the spatial position defines an importance score of the agent.

4. The method of claim 3 , further comprising, for each agent of the plurality of agents:
generating a respective feature representation of the agent, comprising:
generating one or more importance score features of the agent from the output channel;
generating one or more additional features of the agent based on sensor data captured by one or more sensors of the vehicle; and
generating the feature representation of the agent from the importance score features of the agent and the additional features of the agent;

processing the feature representation of the agent using an importance score refining model to generate a refined importance score for the agent that characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle.

5. The method of claim 2 , wherein obtaining the respective values of each motion parameter in the predetermined set of motion parameters for a previous time point comprises one or more of:
obtaining the value of a velocity motion parameter which characterizes a velocity of the vehicle at the previous time point;
obtaining the value of an acceleration motion parameter which characterizes an acceleration of the vehicle at the previous time point; and
obtaining the value of a heading motion parameter which characterizes a heading of the vehicle at the previous time point.

6. The method of claim 2 , wherein the input processed by the importance scoring model further comprises one or more of: (i) a road-graph channel representing a known geometry of the environment, (ii) a vehicle localization channel which represents a spatial position of the vehicle in the environment at a current time point by a vehicle bounding box, and (iii) an auxiliary localization channel which represents respective spatial positions of the plurality of agents in the environment at a current time point by respective bounding boxes.

7. The method of claim 2 , further comprising generating a joint representation of trajectories of the plurality of agents in the environment in the vicinity of the vehicle, wherein the input processed by the importance scoring model further comprises the joint representation of the trajectories of the plurality of agents.

8. The method of claim 7 , wherein:
the joint representation of the trajectories of the plurality of agents in the environment comprises an auxiliary time channel and a respective auxiliary motion channel corresponding to each motion parameter in the predetermined set of motion parameters;
each channel is represented as a two-dimensional array of data values and each data value in each channel corresponds to a respective spatial position in the environment; and
for each particular spatial position in the environment occupied by a particular agent of the plurality of agents at a particular previous time point:
the data value in the auxiliary time channel which corresponds to the particular spatial position defines the particular previous time point; and
for each auxiliary motion channel, the data value in the auxiliary motion channel which corresponds to the particular spatial position defines a value of the motion parameter corresponding to the auxiliary motion channel which characterizes a respective feature of a motion of the particular agent at the particular previous time point.

9. The method of claim 1 , wherein identifying, as high-priority agents, a proper subset of the plurality of agents with the highest importance scores comprises:
identifying, as high-priority agents, a predetermined number of the plurality of agents with the highest importance scores.

10. The method of claim 1 , wherein selecting, for one or more of the agents, a respective prediction model for use in generating data characterizing the agent based on th

Assignees

• Waymo Llc

Inventors

• Ding Kai
• Refaat Khaled
• Ross Stephane

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N20/00

  Machine learning · CPC title

• B60W2556/10

  Historical data · CPC title