Generative artificial intelligence based trajectory simulation

Patent metadata

Field	Value
Publication number	US-2025166364-A1
Application number	US-202318518222-A
Country	US
Kind code	A1
Filing date	Nov 22, 2023
Priority date	Nov 22, 2023
Publication date	May 22, 2025
Grant date	—

Abstract

Devices, systems, and methods a method for simulating a trajectory of an object are described. An example method includes obtaining a context feature representation corresponding to context information, wherein the context information comprises information describing an environment of the object; obtaining a control feature representation corresponding to control information, wherein the control information comprises information that the simulated trajectory needs to satisfy; determining a latent variable using an input encoder based on the context feature representation and the control feature representation; and determining the simulated trajectory by inputting the latent variable, the context feature representation, and the control feature representation into a decoder.

First claim

What is claimed is:

1 . A method for simulating a trajectory of an object, comprising:
obtaining a context feature representation corresponding to context information, wherein the context information comprises information describing an environment of the object; obtaining a control feature representation corresponding to control information, wherein the control information comprises information that the simulated trajectory needs to satisfy; determining a latent variable using an input encoder based on the context feature representation and the control feature representation; and determining the simulated trajectory by inputting the latent variable, the context feature representation, and the control feature representation into a decoder.

2 . The method of claim 1 , wherein obtaining the context feature representation comprises applying the context information into a context encoder.

3 . The method of claim 1 , wherein obtaining the control feature representation comprises applying the control information into a control encoder.

4 . The method of claim 1 , further comprising:
obtaining a concatenated feature representation by concatenating the context feature representation and the control feature representation; and inputting the concatenated feature representation to the input encoder to determine the latent variable.

5 . The method of claim 1 , wherein the context information includes at least one of map information and traffic information.

6 . The method of claim 5 , wherein the traffic information including at least one of information relating to a traffic merge by the object from a ramp or from an adjacent lane or information of neighboring objects in a vicinity of the object.

7 . The method of claim 6 , wherein the map information includes a merge structure of the traffic merge or key points of the merge structure.

8 . The method of claim 1 , wherein the control information includes at least one of a merging distance or aggressiveness of the object.

9 . The method of claim 8 , further comprising determining the merging distance from a continuous distance range.

10 . The method of claim 8 , further comprising determining the merging distance from a plurality of categories.

11 . The method of claim 1 , wherein the input encoder and the decoder constitute a neural network trained based on training datasets that correspond to various traffic scenarios.

12 . The method of claim 11 , wherein the neural network includes a conditional variational autoencoder (CVAE).

13 . The method of claim 1 , wherein at least one of the input encoder or the decoder includes a graph neural network (GNN).

14 . The method of claim 1 , wherein the environment comprises at least one other object.

15 . The method of claim 14 , further comprising:
generating a simulated trajectory according to a pre-determined rule corresponding to a behavior of the at least one other object.

16 . A method for training a neural network configured to simulate a trajectory of an object, the method comprising:
obtaining a plurality of training datasets, each of which includes training context information that includes information describing a training environment of the training object, a training trajectory of a training object, and training operation information that includes information describing a training operation of the training object while the training object traverses the training trajectory, training the neural network based on the plurality of training datasets, wherein the training comprises:
determining, using the neural network being trained, pairs of simulation results each of which includes a simulated latent variable and a corresponding simulated training trajectory and corresponds to one of the plurality of training dataset; and
updating the neural network being trained based on a loss function relating to:
(a) a difference between a distribution of the simulated latent variables and a distribution of latent variables corresponding to the training trajectories, and
(b) a reconstruction loss relating to differences between the simulated training trajectories and corresponding training trajectories, wherein the plurality of training datasets correspond to various traffic scenarios and are balanced such that respective counts of the various traffic scenarios are in a same order.

17 . A system for simulating a trajectory of an object, comprising:
memory storing computer program instructions; and one or more processors configured to execute the computer program instructions to effectuate operations including:
obtaining a context feature representation corresponding to context information, wherein the context information comprises information describing an environment of the object;
obtaining a control feature representation corresponding to control information, wherein the control information comprises information that the simulated trajectory needs to satisfy;
determining a latent variable using an input encoder based on the context feature representation and the control feature representation; and
determining the simulated trajectory by inputting the latent variable, the context feature representation, and the control feature representation into a decoder.

18 . The system of claim 17 , wherein at least one of the context feature representation or the control feature representation comprises information in a form of a feature vector or a subgraph.

19 . The system of claim 17 , wherein the operations further comprise:
obtaining a concatenated feature representation by concatenating the context feature representation and the control feature representation; and inputting the concatenated feature representation to the input encoder to determine the latent variable.

20 . The system of claim 17 , wherein the context information includes at least one of map information and traffic information.

Assignees

• Tusimple Inc

Inventors

• Xiao Hao
• Gan Yiqian
• Ye Xin
• Ye Dongqiangzi
• Miao Jinghao
• Ge Lingting

Classifications

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V10/82 Primary

  using neural networks · CPC title