Play-forward planning and control system for an autonomous vehicle

Patent metadata

Field	Value
Publication number	US-11429107-B2
Application number	US-202016797109-A
Country	US
Kind code	B2
Filing date	Feb 21, 2020
Priority date	Feb 21, 2020
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

A system of creating a simulation to simulate behavior of an autonomous vehicle includes a simulation system having an electronic device and a computer-readable storage medium having one or more programming instructions. When executed, the one or more programming instructions cause the electronic device to identify an event that is to be analyzed, receive from an autonomous vehicle system a first data stream that includes event information from one or more vehicle event log files that corresponds to the event, receive from a vehicle dynamics model a second data stream that includes synthetic event information that corresponds to the event, until a switch point is detected operate in a pure log execution stage, upon detection of the switch point operate in a play-forward execution stage, and cause the new simulation to be executed.

First claim

The invention claimed is:

1. A method of creating a simulation to simulate behavior of an autonomous vehicle, the method comprising:
by an electronic device of a simulation system:
identifying an event that is to be analyzed;
receiving from an autonomous vehicle system a first data stream that comprises event information from one or more vehicle event log files that corresponds to the event, wherein the event information comprises the following obtained by one or more sensors of the autonomous vehicle during occurrence of the event:
first perception information,
first pose information, and
first control information;

receiving from a vehicle dynamics model a second data stream that comprises synthetic event information that corresponds to the event, wherein the synthetic event information is generated by a simulation that simulates operation of the autonomous vehicle in a virtual environment, wherein the synthetic information comprises the following:
second pose information, and
second control information;

operating in a pure log execution stage by:
using at least a portion of the first perception information in a new simulation of the autonomous vehicle,
using at least a portion of the first pose information in the new simulation, and
using at least a portion of the first control information in the new simulation;

detecting a switch point indicating a point where the first perception information begins to diverge from the received synthetic perception data;
upon detection of the switch point, operating in a play-forward execution stage by:
using at least a portion of the first perception information into the new simulation,
using at least a portion of the second pose information in the new simulation, and
using at least a portion of the second control information in the new simulation; and
causing the new simulation to be executed.

2. The method of claim 1 , wherein identifying an event that is to be analyzed comprises receiving from a user an indication of a time period during which the event occurred.

3. The method of claim 1 , wherein the first data stream comprises event information organized in a time sequential fashion.

4. The method of claim 1 , further comprising detecting the switch point by:
identifying an event indicator in the event information, wherein the event indicator indicates the beginning of the event;
determining a timestamp associated with the event indicator;
detecting the switch point at the timestamp.

5. The method of claim 1 , further comprising, upon detection of the switch point, transforming the first perception information by:
moving track position data from the event information into a map reference frame; and
moving data from the map reference frame to a simulation frame.

6. The method of claim 1 , wherein the switch point corresponds to a point at which the simulation diverges from the event as depicted by event information from the one or more vehicle event log files.

7. The method of claim 1 , further comprising not using any portion of the synthetic event information during operation in the pure log execution stage.

8. The method of claim 1 , further comprising not using any portion of the first control information during operation in the play-forward execution stage.

9. The method of claim 1 , further comprising not using any portion of the first pose information during operation in the play-forward execution stage.

10. A system of creating a simulation to simulate behavior of an autonomous vehicle, the system comprising:
a simulation system having an electronic device;
a computer-readable storage medium comprising one or more programming instructions that, when executed, cause the electronic device to:
identify an event that is to be analyzed;
receive from an autonomous vehicle system a first data stream that comprises event information from one or more vehicle event log files that corresponds to the event, wherein the event information comprises the following obtained by one or more sensors of an autonomous vehicle during occurrence of the event:
first perception information,
first pose information, and
first control information;

receive from a vehicle dynamics model a second data stream that comprises synthetic event information that corresponds to the event, wherein the synthetic event information is generated by a simulation that simulates operation of the autonomous vehicle in a virtual environment, wherein the synthetic information comprises the following:
second pose information, and
second control information;

operate in a pure log execution stage by:
using at least a portion of the first perception information in a new simulation of the autonomous vehicle,
using at least a portion of the first pose information in the new simulation, and
using at least a portion of the first control information in the new simulation,

detecting a switch point indicating a point where the first perception information begins to diverge from the received synthetic perception data;
upon detection of the switch point, operate in a play-forward execution stage by:
using at least a portion of the first perception information into the new simulation,
using at least a portion of the second pose information in the new simulation, and
using at least a portion of the second control information in the new simulation; and

cause the new simulation to be executed.

11. The system of claim 10 , wherein the one or more programming instructions that, when executed, cause the electronic device to identify an event that is to be analyzed comprise one or more programming instructions that, when executed, cause the electronic device to receive from a user an indication of a time period during which the event occurred.

12. The system of claim 10 , wherein the first data stream comprises event information organized in a time sequential fashion.

13. The system of claim 10 , wherein the computer-readable storage medium further comprises one or more programming instructions that, when executed, cause the electronic device to detect the switch point by:
identifying an event indicator in the event information, wherein the event indicator indicates the beginning of the event;
determining a timestamp associated with the event indicator;
detecting the switch point at the timestamp.

14. The system of claim 10 , wherein the computer-readable storage medium further comprises one or more programming instructions that, when executed, cause the electronic device to, upon detection of the switch point, transform the first perception information by:
moving track position data from the event information into a map reference frame; and
moving data from the map reference frame to a simulation frame.

15. The system of claim 10 , wherein the switch point corresponds to a point at which the simulation diverges from the event as depicted by event information from the one or more vehicle event log files.

16. The system of claim 10 , wherein the computer-readable storage medium further comprises one or more programming instructions that, when executed, cause the electronic device to not use any portion of the synthetic event information during operation in the pure log execution stage.

17. The system of claim 10 , wherein the computer-readable storage medium further comprises one or more programming instructions that, when executed, cause the electronic device to not using any portion of the first control information during operation in the play-forward execution stage.

18. The system of claim 10

Assignees

• Argo Ai Llc

Inventors

• Ackenhausen Thomas Carl
• Carmody Patrick Michael
• Costa Albert

Classifications

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3696 Primary

  Methods or tools to render software testable · CPC title

• G05D1/0221 Primary

  involving a learning process · CPC title

• G05D2201/0213

  Physics · mapped topic

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title