Method for learning an explainable trajectory generator using an automaton generative network

Patent metadata

Field	Value
Publication number	US-12024203-B2
Application number	US-202117372083-A
Country	US
Kind code	B2
Filing date	Jul 9, 2021
Priority date	Jul 9, 2021
Publication date	Jul 2, 2024
Grant date	Jul 2, 2024

Abstract

A method of generating an output trajectory of an ego vehicle is described. The method includes extracting high-level features from a bird-view image of a traffic environment of the ego vehicle. The method also includes generating, using an automaton generative network, an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features. The method further includes generating the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment.

First claim

What is claimed is:

1. A method of generating an output trajectory of an ego vehicle, the method comprising:
extracting high-level features from a bird-view image of a traffic environment of the ego vehicle;
generating, using an automaton generative network, an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features, in which the automaton comprises a predicate deterministic finite state automaton (DFA) having learned weights; and
generating the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment.

2. The method of claim 1 , further comprising performing a vehicle control action to maneuver the ego vehicle along the output trajectory.

3. The method of claim 1 , in which the high-level features comprise agent positions, velocities, and lane representations shown in the bird-view image.

4. The method of claim 1 , further comprises learning from continuous actions and continuous trajectories.

5. A method of generating an output trajectory of an ego vehicle, the method comprising:
extracting high-level features from a bird-view image of a traffic environment of the ego vehicle;
generating, using an automaton generative network, an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features by calculating, using the automaton generative network, an alphabet vector corresponding to the automaton state distribution according to the high-level features of the bird-view image; and
generating the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment.

6. A method of generating an output trajectory of an ego vehicle, the method comprising:
extracting high-level features from a bird-view image of a traffic environment of the ego vehicle;
generating, using an automaton generative network, an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features;
generating the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment;
bootstrapping, by the automaton generative network, according to logic priors; and
training an automaton structure of the automaton generative network according to driving data.

7. The method of claim 6 , in which the logic priors comprise temporal logic formulas defining the behavior of the ego vehicle in the traffic environment.

8. A non-transitory computer-readable medium having program code recorded thereon for generating an output trajectory of an ego vehicle, the program code being executed by a processor and comprising:
program code to extract high-level features from a bird-view image of a traffic environment of the ego vehicle;
program code to generate, using an automaton generative network, an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features, in which the automaton comprises a predicate deterministic finite state automaton (DFA) having learned weights; and
program code to generate the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment.

9. The non-transitory computer-readable medium of claim 8 , further comprising program code to perform a vehicle control action to maneuver the ego vehicle along the output trajectory.

10. The non-transitory computer-readable medium of claim 8 , in which the program code to generate the automaton comprises program code to calculate, using the automaton generative network, an alphabet vector corresponding to the automaton state distribution according to the high-level features of the bird-view image.

11. The non-transitory computer-readable medium of claim 8 , in which the high-level features comprise agent positions, velocities, and lane representations shown in the bird-view image.

12. The non-transitory computer-readable medium of claim 8 , further comprises program code to learn from continuous actions and continuous trajectories.

13. The non-transitory computer-readable medium of claim 12 , in which the program code to learn comprises:
program code to bootstrap, by the automaton generative network, according to logic priors; and
program code to train an automaton structure of the automaton generative network according to driving data.

14. The non-transitory computer-readable medium of claim 13 , in which the logic priors comprise temporal logic formulas defining the behavior of the ego vehicle in the traffic environment.

15. A system for generating an output trajectory of an ego vehicle, the system comprising:
a vehicle perception module to extract high-level features from a bird-view image of a traffic environment of the ego vehicle;
an automaton generative network to generate an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features, in which the automaton comprises a predicate deterministic finite state automaton (DFA) having learned weights; and
a trajectory generation module to generate the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment.

16. The system of claim 15 , further comprising a vehicle control selection module to select a vehicle control action to maneuver the ego vehicle along the output trajectory.

17. The system of claim 15 , in which the high-level features comprise agent positions, velocities, and lane representations shown in the bird-view image.

Assignees

• Toyota Res Inst Inc
• Massachusetts Inst Technology
• Toyota Res Institute

Inventors

• Li Xiao
• Araki Brandon
• Karaman Sertac
• Rus Daniela
• Rosman Guy
• Gilitschenski Igor
• Vasile Cristian-Ioan

Classifications

• B60W2554/4041

  Position · CPC title

• B60W2554/4043

  Lateral speed · CPC title

• B60W2554/4042

  Longitudinal speed · CPC title

• B60W2554/408

  Traffic behavior, e.g. swarm · CPC title

• G06N3/08

  Learning methods · CPC title