Systems and methods for comparing driving performance for simulated driving

Patent metadata

Field	Value
Publication number	US-2020005662-A1
Application number	US-201816025799-A
Country	US
Kind code	A1
Filing date	Jul 2, 2018
Priority date	Jul 2, 2018
Publication date	Jan 2, 2020
Grant date	—

Abstract

This disclosure relates to a system that determines driving performance by a vehicle operator for simulated driving of a simulated vehicle in a simulation engine. Individual vehicle event scenarios correspond to vehicle events. Individual simulation scenarios correspond to individual vehicle event scenarios. A vehicle operator, e.g., an autonomous driving algorithm, operates the simulated vehicle in the simulation engine for a set of simulation scenarios. One or more metrics quantify the performance of the vehicle operator based on simulated results.

First claim

What is claimed is:

1 . A system configured to determine driving performance by a vehicle operator for simulated driving of a simulated vehicle in a simulation engine, the system comprising:
electronic storage configured to electronically store information; and one or more processors configured via machine-readable instructions to:
obtain a stored set of vehicle event scenarios that correspond to vehicle events, wherein individual vehicle events are associated with physical surroundings of individual vehicles around the times of the individual vehicle events, wherein a first vehicle event scenario is associated with a first set of circumstances that is based on a first set of physical surroundings of a first vehicle around the time a first vehicle event occurred, wherein the first vehicle event scenario has a scenario time period that begins prior to an occurrence of a potential vehicle event;
create a set of simulation scenarios that are suitable for use by the simulation engine, wherein individual ones of the set of simulation scenarios correspond to individual ones of the stored set of vehicle event scenarios, wherein individual ones of the set of simulation scenarios mimic the circumstances associated with a corresponding vehicle event scenario, such that a first simulation scenario mimics the first set of circumstances associated with the first vehicle event scenario, wherein the simulated vehicle is based on the first vehicle;
establish a link between the vehicle operator and the simulation engine;
run the set of simulation scenarios in the simulation engine, wherein the simulated vehicle is operated by the vehicle operator;
determine one or more metrics that quantify a performance of the vehicle operator in running the set of simulation scenarios; and
store and/or transfer the determined one or more metrics.

2 . The system of claim 1 , wherein the potential vehicle vent corresponds to the first vehicle event.

3 . The system of claim 1 , wherein the vehicle operator is an autonomous driving algorithm.

4 . The system of claim 1 , wherein the one or more processors are further configured via machine-readable instructions to:
establish a second link between a second vehicle operator and the simulation engine; run the set of simulation scenarios in the simulation engine, wherein the simulated vehicle is operated by the second vehicle operator; determine a second set of one or more metrics that quantify a second performance of the second vehicle operator in running the set of simulation scenarios; compare the determined one or more metrics with the determined second set of one or more metrics; and determine whether the vehicle operator performed better than the second vehicle operator, wherein the determination is based on the comparison.

5 . The system of claim 4 , wherein the second vehicle operator is a benchmark autonomous driving algorithm.

6 . The system of claim 1 , wherein individual ones of the stored set of vehicle event scenarios correspond to detected vehicle events in real life.

7 . The system of claim 1 , wherein the link includes a communication link between the vehicle operator and the simulation engine, wherein the communication link provides the vehicle operator with control over operations of the simulated vehicle.

8 . The system of claim 3 , wherein the set of simulation scenarios in the simulation engine is run at faster-than-real-time.

9 . The system of claim 1 , wherein one of the one or more metrics is reduced responsive to an individual one of the set of simulation scenarios resulting in a preventable accident.

10 . The system of claim 1 , wherein one of the one or more metrics represents a ratio of a first subset of the set of simulation scenarios and a second subset of the set of simulation scenarios, wherein the first subset resulted in preventable accidents, and wherein the second subset completed without preventable accidents.

11 . A method for determining driving performance by a vehicle operator for simulated driving of a simulated vehicle in a simulation engine, the method comprising:
obtaining a stored set of vehicle event scenarios that correspond to vehicle events, wherein individual vehicle events are associated with physical surroundings of individual vehicles around the times of the individual vehicle events, wherein a first vehicle event scenario is associated with a first set of circumstances that is based on a first set of physical surroundings of a first vehicle around the time a first vehicle event occurred, wherein the first vehicle event scenario has a scenario time period that begins prior to an occurrence of a potential vehicle event; creating a set of simulation scenarios that are suitable for use by the simulation engine, wherein individual ones of the set of simulation scenarios correspond to individual ones of the stored set of vehicle event scenarios, wherein individual ones of the set of simulation scenarios mimic the circumstances associated with a corresponding vehicle event scenario, such that a first simulation scenario mimics the first set of circumstances associated with the first vehicle event scenario, wherein the simulated vehicle is based on the first vehicle; establishing a link between the vehicle operator and the simulation engine; running the set of simulation scenarios in the simulation engine, wherein the simulated vehicle is operated by the vehicle operator; determining one or more metrics that quantify a performance of the vehicle operator in running the set of simulation scenarios; and storing and/or transferring the determined one or more metrics.

12 . The method of claim 11 , wherein the potential vehicle vent corresponds to the first vehicle event.

13 . The method of claim 11 , wherein the vehicle operator is an autonomous driving algorithm.

14 . The method of claim 11 , further comprising:
establishing a second link between a second vehicle operator and the simulation engine; running the set of simulation scenarios in the simulation engine, wherein the simulated vehicle is operated by the second vehicle operator; determining a second set of one or more metrics that quantify a second performance of the second vehicle operator in running the set of simulation scenarios; comparing the determined one or more metrics with the determined second set of one or more metrics; and determining whether the vehicle operator performed better than the second vehicle operator, wherein the determination is based on the comparison.

15 . The method of claim 14 , wherein the second vehicle operator is a benchmark autonomous driving algorithm.

16 . The method of claim 11 , wherein individual ones of the stored set of vehicle event scenarios correspond to detected vehicle events in real life.

17 . The method of claim 11 , wherein the link includes a communication link between the vehicle operator and the simulation engine, wherein the communication link provides the vehicle operator with control over operations of the simulated vehicle.

18 . The method of claim 13 , wherein the set of simulation scenarios in the simulation engine is run at faster-than-real-time.

19 . The method of claim 11 , wherein one of the one or more metrics is reduced responsive to an individual one of the set of simulation scenarios resulting in a preventable accident.

20 . The method of claim 11 , wherein one of the one or more metrics represents a ratio of a first subset of the set of simulation scenarios and a second subse

Assignees

• Smartdrive Systems Inc

Inventors

• Palmer Jason
• Ghanbari Reza
• Brookins Nicholas Shayne
• Sljivar Slaven
• Freitas Mark
• Parshall Barry James
• Deninger Daniel Andrew
• Shahrasbi Behzad
• Ziyadi Mojtaba

Classifications

• G09B9/052 Primary

  characterised by provision for recording or measuring trainee's performance (devices for psychotechnics, e.g. for vehicle drivers A61B5/16, A61B5/18) · CPC title

• G08G1/167

  Driving aids for lane monitoring, lane changing, e.g. blind spot detection · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W40/09

  Driving style or behaviour · CPC title