Systems and methods for comparing driving performance for simulated driving

Patent metadata

Field	Value
Publication number	US-12008922-B2
Application number	US-201816025799-A
Country	US
Kind code	B2
Filing date	Jul 2, 2018
Priority date	Jul 2, 2018
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

This disclosure relates to a system that determines driving performance by a vehicle operator for simulated driving of a simulated vehicle in a simulation engine. Individual vehicle event scenarios correspond to vehicle events. Individual simulation scenarios correspond to individual vehicle event scenarios. A vehicle operator, e.g., an autonomous driving algorithm, operates the simulated vehicle in the simulation engine for a set of simulation scenarios. One or more metrics quantify the performance of the vehicle operator based on simulated results.

First claim

What is claimed is:

1. A system configured to determine driving performance by a vehicle operator and by a second vehicle operator for simulated driving of a simulated vehicle in a simulation engine, wherein the vehicle operator and the second vehicle operator are included in a set of vehicle operators, wherein the simulated driving is based on real-world operation of real-world vehicles, the system comprising:
electronic storage configured to electronically store information; and
one or more processors configured via machine-readable instructions to:
determine one or more vehicle parameters of individual vehicles, wherein the one or more vehicle parameters are determined directly based on output signals from at least two different sensors, wherein the one or more vehicle parameters are determined multiple times in an ongoing manner during operation of the vehicle;
obtain a stored set of vehicle event scenarios that correspond to vehicle events that have occurred in the real world at particular times during the real-world operation of the real-world vehicles, wherein individual vehicle events are associated with physical surroundings of individual vehicles around the particular times the individual vehicle events occurred, wherein individual vehicle events are detected based on comparing the one or more vehicle parameters of individual vehicles with one or more thresholds, wherein a first vehicle event scenario is associated with a first set of circumstances that is based on a first set of physical surroundings of a first vehicle around a particular time a first vehicle event occurred, wherein the first vehicle event scenario has a scenario time period that begins prior to an occurrence of a potential vehicle event;
create a set of simulation scenarios that are suitable for use by the simulation engine, wherein individual ones of the set of simulation scenarios correspond to individual ones of the stored set of vehicle event scenarios, wherein individual ones of the set of simulation scenarios mimic circumstances associated with a corresponding vehicle event scenario, such that a first simulation scenario mimics the first set of circumstances associated with the first vehicle event scenario, wherein the simulated vehicle is based on the first vehicle;
establish a communication link between the vehicle operator and the simulation engine, wherein the vehicle operator is an autonomous driving algorithm that controls operations of the simulated vehicle autonomously through the communication link;
run the set of simulation scenarios in the simulation engine, wherein the simulated vehicle interacts with the vehicle operator and is operated by the vehicle operator based on input received from the autonomous driving algorithm;
detect, by the simulation engine during the running of the set of simulation scenarios, one or more simulated vehicle events that have occurred to the simulated vehicle as the autonomous driving algorithm controlled the operations of the simulated vehicle;
determine one or more metrics that quantify a performance of the vehicle operator in running the set of simulation scenarios, wherein the determination of the one or more metrics is based on the one or more simulated vehicle events as detected;
establish a second communication link between the second vehicle operator and the simulation engine, wherein the second vehicle operator is a second autonomous driving algorithm that controls operations of the simulated vehicle autonomously through the second communication link;
run the set of simulation scenarios in the simulation engine, wherein the simulated vehicle interacts with the second vehicle operator and is operated by the second vehicle operator based on second input received from the second autonomous driving algorithm;
detect, by the simulation engine during the running of the set of simulation scenarios, a second set of simulated vehicle events that have occurred to the simulated vehicle as the second autonomous driving algorithm controlled the operations of the simulated vehicle;
determine a second set of metrics that quantify a second performance of the second vehicle operator running the set of simulation scenarios, wherein the determination of the second set of metrics is based on the second set of simulated vehicle events as detected;
compare the performance of the vehicle operator with the second performance of the second vehicle operator, wherein comparison is based on the one or more metrics and the second set of metrics;
based on the comparison, determine whether the vehicle operator or the second vehicle operator performed best in running the set of simulation scenarios; and
generate a report based on the comparison and transfer the report to one or more users.

2. The system of claim 1 , wherein the potential vehicle event corresponds to the first vehicle event.

3. The system of claim 1 , wherein the second vehicle operator is a benchmark autonomous driving algorithm.

4. The system of claim 1 , wherein individual ones of the stored set of vehicle event scenarios correspond to detected vehicle events in real life.

5. The system of claim 1 . wherein the communication link provides the vehicle operator with control over the operations of the simulated vehicle.

6. The system of claim 1 , wherein the simulation engine is computer-implemented and configured by a set of machine-readable instructions, and wherein the set of simulation scenarios in the simulation engine is run at faster-than-real-time.

7. The system of claim 1 , wherein one of the one or more metrics is reduced responsive to an individual one of the set of simulation scenarios resulting in an accident.

8. The system of claim 1 , wherein the one or more processors are further configured to take an action based on the comparison of the performance of the vehicle operator with the second performance of the second vehicle operator.

9. A method for determining driving performance by a vehicle operator and by a second vehicle operator for simulated driving of a simulated vehicle in a simulation engine, wherein the vehicle operator and the second vehicle operator are included in a set of vehicle operators, wherein the simulated driving is based on real-world operation of real-world vehicles, the method comprising:
determining one or more vehicle parameters of individual vehicles, wherein the one or more vehicle parameters are determined directly based on output signals from at least two different sensors, wherein the one or more vehicle parameters are determined multiple times in an ongoing manner during operation of the vehicle;
obtaining a stored set of vehicle event scenarios that correspond to vehicle events that have occurred in the real world at particular times during the real-world operation of the real-world vehicles, wherein individual vehicle events are associated with physical surroundings of individual vehicles around the particular times the individual vehicle events occurred, wherein individual vehicle events are detected based on comparing the one or more vehicle parameters of individual vehicles with one or more thresholds, wherein a first vehicle event scenario is associated with a first set of circumstances that is based on a first set of physical surroundings of a first vehicle around a particular time a first vehicle event occurred, wherein the first vehicle event scenario has a scenario time period that begins prior to an occurrence of a potential vehicle event;
creating a set of simulation scenarios that are suitable for use by the simulation engine, wherein individual ones of the set of simulation scenarios correspond to individual ones of the stored set of vehicle event scenarios, wherein individual ones of the set of simulat

Assignees

• Smartdrive Systems Inc

Inventors

• Palmer Jason
• Ghanbari Reza
• Brookins Nicholas Shayne
• Sljivar Slaven
• Freitas Mark
• Parshall Barry James
• Deninger Daniel Andrew
• Shahrasbi Behzad
• Ziyadi Mojtaba

Classifications

• G08G1/167

  Driving aids for lane monitoring, lane changing, e.g. blind spot detection · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W40/09

  Driving style or behaviour · CPC title

• G09B9/052 Primary

  characterised by provision for recording or measuring trainee's performance (devices for psychotechnics, e.g. for vehicle drivers A61B5/16, A61B5/18) · CPC title