Simulation-based training of an autonomous vehicle

Patent metadata

Field	Value
Publication number	US-12422791-B2
Application number	US-202117345738-A
Country	US
Kind code	B2
Filing date	Jun 11, 2021
Priority date	Jun 12, 2020
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

A controller for an autonomous vehicle is trained using simulated paths on a roadway and simulated observations that are formed by transforming images previously acquired on similar paths on that roadway. Essentially an unlimited number of paths may be simulated, enabling optimization approaches including reinforcement learning to be applied to optimize the controller.

First claim

What is claimed is:

1. A method comprising:
simulating a simulated agent's traversal of a simulated path in a real environment, including simulating an autonomous vehicle in a simulated traversal of a roadway;
prior to simulation of the simulated agent's traversal of the simulated path, obtaining a plurality of observations acquired by real entities as they traversed real paths, each observation including an image acquired when a corresponding real entity was in a location corresponding to said image on one of the real paths, wherein at least some of the real paths traverse the roadway, and wherein the real entities comprise a human-operated real vehicle that traversed one of the real paths;
wherein the simulating of the simulated agent's traversal comprises simulating a simulated agent's view to form simulated observations, including simulating a first image for a first location on the simulated path, the simulating of the first image including selecting a second image acquired by one of the real entities at a second location different than the first location, and transforming the second image to form the first image according to a relationship of the first location and the second location, wherein the second image comprises a two-dimensional image and the first image comprises a two-dimensional image, wherein transforming the second image to form the first image comprises forming a second three-dimensional image from the second two-dimensional image, forming a first three-dimensional image from the second three-dimensional image according to the relationship of the first location and the second location, and forming the first two-dimensional image from the first three-dimensional image;
wherein the simulating of the simulated agent's traversal further comprises processing the simulated observations according to a control policy to yield control actions, and causing the simulated agent to transverse the simulated path according to the control actions;
wherein the method further comprises updating the control policy according to the simulated agent's traversal of the simulated path, including updating the policy according to the simulated agent's traversal of a plurality of simulated paths according to simulations of the simulated agent's view in a plurality of environmental conditions.

2. The method of claim 1 , wherein each image comprises a photographic depiction of the environment.

3. The method of claim 1 , wherein selecting the second image includes selecting said image according to a match of the first location and the second location.

4. The method of claim 3 , wherein the first location and the second location each includes a corresponding position and orientation, and the match of said locations is based on a degree of perturbation of the second location to yield the first location.

5. The method of claim 1 , wherein forming the second three-dimensional image from the second two-dimensional image includes using a second depth map corresponding to the second two-dimensional image.

6. The method of claim 5 , wherein forming the second three-dimensional image further comprises determining the second depth map from the second two-dimensional image.

7. The method of claim 1 , wherein processing the simulated observations to yield control actions includes at least one of determining at least one of steering and speed commands.

8. The method of claim 1 , wherein processing the simulated observations includes processing said observations using a convolutional neural network to yield the control actions.

9. The method of claim 1 , wherein updating the control policy includes applying a Reinforcement Learning (RL) procedure to update a stochastic policy.

10. The method of claim 1 , wherein the plurality of environmental conditions comprises at least one of different weather conditions, different times of day, and different road types.

11. The method of claim 1 , wherein obtaining the plurality of observations acquired by the real entities as they traversed the real paths comprise acquiring images in a plurality of different environmental conditions.

12. A computer-readable medium comprising instructions stored thereon, the instructions when executed by a data processor cause operations including:
simulating a simulated agent's traversal of a simulated path in a real environment, including simulating an autonomous vehicle in a simulated traversal of a roadway;
prior to simulation of the simulated agent's traversal of the simulated path, obtaining a plurality of observations acquired by real entities as they traversed real paths, each observation including an image acquired when a corresponding real entity was in a location corresponding to said image on one of the real paths, wherein at least some of the real paths traverse the roadway, and wherein the real entities comprise a human-operated real vehicle that traversed one of the real paths;
wherein the simulating of the simulated agent's traversal comprises simulating a simulated agent's view to form simulated observations, including simulating a first image for a first location on the simulated path, the simulating of the first image including selecting a second image acquired by one of the real entities at a second location different than the first location, and transforming the second image to form the first image according to a relationship of the first location and the second location, wherein the second image comprises a two-dimensional image and the first image comprises a two-dimensional image, wherein transforming the second image to form the first image comprises forming a second three-dimensional image from the second two-dimensional image, forming a first three-dimensional image from the second three-dimensional image according to the relationship of the first location and the second location, and forming the first two-dimensional image from the first three-dimensional image;
wherein the simulating of the simulated agent's traversal further comprises processing the simulated observations according to a control policy to yield control actions, and causing the simulated agent to transverse the simulated path according to the control actions;
wherein operations futher include updating the control policy according to the simulated agent's traversal of the simulated path, including updating the policy according to the simulated agent's traversal of a plurality of simulated paths according to simulations of the simulated agent's view in a plurality of environmental conditions.

Assignees

• Massachusetts Inst Technology

Inventors

• Rus Daniela
• Karaman Sertac
• Gilitschenski Igor
• Amini Alexander
• Moseyko Julia
• Phillips Jacob

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06F16/50

  of still image data · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06T3/00

  Geometric image transformations in the plane of the image · CPC title

• G06T15/40

  Hidden part removal · CPC title