Machine-learned cost estimation in tree search trajectory generation for vehicle control

Patent metadata

Field	Value
Publication number	US-12311981-B2
Application number	US-202218084419-A
Country	US
Kind code	B2
Filing date	Dec 19, 2022
Priority date	Dec 19, 2022
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

A machine-learned architecture for estimating the cost determined by a cost function for a prediction node of a tree search for exploring potential paths for controlling a vehicle may include two portions: a set up portion that includes models trained to process static data and a second portion that processes dynamic object data. The respective portions of the architecture may comprise various models that determine intermediate outputs that may be projected into a space associated with estimated cost. That estimated cost may identify an estimate of an output of the cost function for paths that are based on a particular prediction node of the tree search.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving environment state data indicating static characteristics of an environment associated with a vehicle;
receiving dynamic object data indicating at least one of a historical or a current state of at least one of a dynamic object or the vehicle;
receiving prediction node data indicating one or more actions performed by the vehicle to reach a future state indicated by a prediction node of a tree search, the future state comprising a future vehicle state and at least one of a predicted environment state, or a predicted dynamic object state;
determining, by a first machine-learned model and based at least in part on the environment state data, environment features;
determining, by a second machine-learned model and based at least in part on the dynamic object data, dynamic features;
initializing a third machine-learned model based at least in part on the dynamic features, wherein the first machine-learned model, the second machine-learned model, and the third machine-learned model are distinct and have different types of inputs;
determining, by the third machine-learned model and based at least in part on the environment features and the prediction node data, a first output;
aggregating, by an encoder, the first output and the dynamic features as an encoded output;
determining, by a decoder and based at least in part on the encoded output, an estimated cost; and
determining a trajectory for controlling the vehicle based at least in part on the estimated cost.

2. The system of claim 1 , wherein:
the estimated cost is associated with the prediction node indicating a predicted state of the environment, the dynamic object, and the vehicle at a future time; and
the estimated cost estimates a total cost determined by a cost function for one or more candidate actions that are potentially to be taken by the vehicle at the future time based on the predicted state of the environment to reach a termination point.

3. The system of claim 2 , wherein the operations further comprise determining the total cost, determining the cost comprises determining a cost associated with a first candidate action of the one or more candidate actions, and the cost function determines the cost based at least in part on at least one of:
a proximity sub-cost based at least in part on a distance the first candidate action to bring the vehicle from an object location or predicted object location;
a safety sub-cost associated with the first candidate action;
a comfort sub-cost associated with the first candidate action; or
a progress sub-cost associated with the first candidate action.

4. The system of claim 2 , wherein the operations further comprise training at least one of the first machine-learned model, the second machine-learned model, the third machine-learned model, the encoder, or the decoder based at least in part on:
determining a first cost based at least in part on the cost function and a candidate action associated with the prediction node;
determining a difference between the first cost and the estimated cost; and
altering a parameter of at least one of the first machine-learned model, the second machine-learned model, the third machine-learned model, the encoder, or the decoder to reduce the difference.

5. The system of claim 1 , wherein initializing the third machine-learned model comprises:
determining a portion of the dynamic features associated with the dynamic object; and
adding the portion of the dynamic features to at least one of a node or a hidden layer of the third machine-learned model.

6. The system of claim 1 , wherein the first machine-learned model, the second machine-learned model, and the third machine-learned model are portions of a single machine-learned model.

7. A method comprising:
receiving environment state data indicating static characteristics of an environment associated with a vehicle;
receiving dynamic object data indicating at least one of a historical or a current state of at least one of a dynamic object or the vehicle;
determining, by a first machine-learned model and based at least in part on the environment state data, environment features;
determining, by a second machine-learned model and based at least in part on the dynamic object data, dynamic features;
initializing a third machine-learned model based at least in part on the dynamic features, wherein the first machine-learned model, the second machine-learned model, and the third machine-learned model are distinct and have different types of inputs;
determining, by the third machine-learned model and based at least in part on the environment features and prediction node data indicating one or more actions performed by the vehicle to reach a future state indicated by a prediction node of a tree search, a first output;
aggregating, by an encoder, the first output and the dynamic features as an encoded output;
determining, by a decoder and based at least in part on the encoded output, an estimated cost; and
determining a trajectory for controlling the vehicle based at least in part on the estimated cost.

8. The method of claim 7 , wherein:
the estimated cost is associated with the prediction node indicating a predicted state of the environment, the dynamic object, and the vehicle at a future time; and
the estimated cost estimates a total cost output by a cost function for one or more candidate actions that are potentially to be taken by the vehicle at the future time based on the predicted state of the environment to reach a termination point.

9. The method of claim 8 , further comprising: determining the total cost, determining the cost comprises determining a cost associated with a first candidate action of the one or more candidate actions, and the cost function determines the cost based at least in part on at least one of:
a proximity sub-cost based at least in part on a distance the first candidate action to bring the vehicle from an object location or predicted object location;
a safety sub-cost associated with the first candidate action;
a comfort sub-cost associated with the first candidate action; or
a progress sub-cost associated with the first candidate action.

10. The method of claim 8 , further comprising training at least one of the first machine-learned model, the second machine-learned model, the third machine-learned model, the encoder, or the decoder based at least in part on:
determining a first cost based at least in part on the cost function and a candidate action associated with the prediction node;
determining a difference between the first cost and the estimated cost; and
altering a parameter of at least one of the first machine-learned model, the second machine-learned model, the third machine-learned model, the encoder, or the decoder to reduce the difference.

11. The method of claim 8 , wherein:
the prediction node is a first prediction node among a plurality of prediction nodes of the tree search and the estimated cost is a first estimated cost associated with the first prediction node; and
the method further comprises:
determining multiple estimated costs associated with the plurality of prediction nodes, and
determining that the first estimated cost is a lowest estimated cost from among the multiple estimated costs; and
determining the trajectory comprises:
determining, by the cost function, a respective cost for each of a different series of candidate actions that are based at least in part on a corresponding

Assignees

• Zoox Inc

Inventors

• Chang Yan
• Huang Aaron
• Schleede Peter Scott
• Linscott Gary
• Kobilarov Marin
• Pronovost Ethan Miller
• Sun Ke
• Xie Xiangyu

Classifications

• B60W30/12

  Lane keeping · CPC title

• B60W2554/20

  Static objects · CPC title

• B60W30/18163

  Lane change; Overtaking manoeuvres · CPC title

• B60W2554/40

  Dynamic objects, e.g. animals, windblown objects · CPC title

• G06N20/00 Primary

  Machine learning · CPC title