Testing autonomous cars

Patent metadata

Field	Value
Publication number	US-10909629-B1
Application number	US-201715433166-A
Country	US
Kind code	B1
Filing date	Feb 15, 2017
Priority date	Feb 15, 2016
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

Systems and methods are disclosed for determining the efficacy of an autonomous vehicle driving system in exhibiting safe driving behavior during test track scenarios. Testing parameters eliciting autonomous vehicle driving system behavior may be chosen for the test track scenario. The autonomous vehicle driving system may be tested in the test track and autonomous vehicle driving system behavior may be observed in response to the presentations of testing parameters. A safe driving score may be calculated for the autonomous vehicle driving system based on measured autonomous vehicle performance and operational data generated responsive to the presentation of testing parameters in the test track. A benchmark insurance premium may be calculated for an autonomous vehicle driving system based on the calculated safe driving score.

First claim

We claim:

1. A method comprising:
determining, by a driving analysis computing device, first testing parameters associated with a first testing track, wherein the first testing parameters comprise first testing input data representative of vehicle sensor data;
testing an autonomous vehicle driving system on the first testing track by:
sending, to a vehicle computing device associated with the autonomous vehicle driving system, the first testing input data representative of vehicle sensor data; and
receiving, from the vehicle computing device, first autonomous vehicle driving data representative of virtual driving responses generated in response to the first testing input data representative of vehicle sensor data;

determining, based on the first testing input data representative of vehicle sensor data and the first autonomous vehicle driving data representative of virtual driving responses, a first safe driving score for the autonomous vehicle driving system on the first testing track;
comparing the first safe driving score for the autonomous vehicle driving system to a safe driving score for a driver-operated vehicle, wherein the safe driving score for the driver-operated vehicle is calculated using testing parameters correlating with the first testing parameters;
determining, based on the safe driving score for the driver-operated vehicle, a base driver insurance premium;
determining, based on the first safe driving score and the base driver insurance premium, a first benchmark insurance premium for the autonomous vehicle driving system; and
applying the first benchmark insurance premium to one or more insurance policies for one or more autonomous vehicles associated with the autonomous vehicle driving system.

2. The method of claim 1 , further comprising:
responsive to determining the first benchmark insurance premium, determining, by the driving analysis computing device, second testing parameters associated with a second testing track;
testing the autonomous vehicle driving system on the second testing track by:
presenting, to a first vehicle comprising the vehicle computing device and while the first vehicle is driving on the second testing track, the second testing parameters; and
receiving, from the vehicle computing device, second autonomous vehicle driving data representative of actual driving responses generated in response to the second testing parameters;

determining, based on the second testing parameters and the second autonomous vehicle driving data, a second safe driving score for the autonomous vehicle driving system on the second testing track; and
determining, based on the second safe driving score, a second benchmark insurance premium for the autonomous vehicle driving system.

3. The method of claim 2 , further comprising:
responsive to determining the second benchmark insurance premium, determining, by the driving analysis computing device, third testing parameters associated with a third testing track;
testing the autonomous vehicle driving system on the third testing track by:
sending, to the vehicle computing device of the first vehicle, the third testing parameters and instructions for the first vehicle to operate in accordance with the third testing parameters; and
receiving, from the vehicle computing device, third autonomous vehicle driving data representative of actual driving responses generated in response to the third testing parameters;

determining, based on the third testing parameters and the third autonomous vehicle driving data, a third safe driving score for the autonomous vehicle driving system on the third testing track; and
determining, based on the third safe driving score, a third benchmark insurance premium for the autonomous vehicle driving system.

4. The method of claim 3 , further comprising:
determining, based on the first, second, and third safe driving scores associated with the autonomous vehicle driving system, an average safe driving score for the autonomous vehicle driving system; and
calculating, based on the average safe driving score, an average benchmark insurance premium for the autonomous vehicle driving system.

5. The method of claim 1 , wherein the first testing track is a virtual testing track.

6. The method of claim 2 , wherein the second testing track is a closed testing track, and wherein the determined second testing parameters for the closed testing track are controllable and variable components.

7. The method of claim 3 , wherein the third testing track is an open testing track, and wherein the determined third testing parameters comprise at least one of: a specific road segment, a predetermined time period, or one or more testing conditions, and
wherein the instructions for the first vehicle to operate in accordance with the third testing parameters comprise instructions for the first vehicle to drive on the specific road segment during the predetermined time period while testing the autonomous vehicle driving system on the third testing track.

8. The method of claim 3 , wherein the autonomous vehicle driving system is prohibited from progressing from the first testing track to the second testing track when the first safe driving score is below a predetermined threshold, and
wherein the autonomous vehicle driving system is prohibited from progressing from the second testing track to the third testing track when the second safe driving score is below a predetermined threshold.

9. A computing device, comprising:
at least one processor; and
memory storing computer-readable instructions that, when executed by the at least one processor, cause the computing device to:
determine first testing parameters associated with a first testing track, wherein the first testing parameters comprise first testing input data representative of vehicle sensor data;
test an autonomous vehicle driving system on the first testing track by:
sending, to a vehicle computing device associated with the autonomous vehicle driving system, the first testing input data representative of vehicle sensor data; and
receiving, from the vehicle computing device, first autonomous vehicle driving data representative of virtual driving responses generated in response to the first testing input data representative of vehicle sensor data;

determine, based on the first testing input data representative of vehicle sensor data and the first autonomous vehicle driving data representative of virtual driving, responses a first safe driving score for the autonomous vehicle driving system on the first testing track;
compare the first safe driving score for the autonomous vehicle driving system to a safe driving score for a driver-operated vehicle, wherein the safe driving score for the driver-operated vehicle is calculated using testing parameters correlating with the first testing parameters;
determine, based on the safe driving score for the driver-operated vehicle, a base driver insurance premium;
determine, based on the first safe driving score and the base driver insurance premium, a first benchmark insurance premium for the autonomous vehicle driving system; and
apply the first benchmark insurance premium to one or more insurance policies for one or more autonomous vehicles associated with the autonomous vehicle driving system.

10. The computing device of claim 9 , wherein the memory stores further computer-readable instructions that, when executed by the at least one processor, cause the computing device to:
responsive to determining the first benchmark insurance premium, determine second testing parameters associated with a second testing track;
test the autonomous vehicle driving system on the second testing track by:
presenting, to a fi

Assignees

• Allstate Insurance Co

Inventors

• Madigan Regina
• Polisson William
• Slusar Mark V

Classifications

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G07C5/0808

  Diagnosing performance data (testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005) · CPC title

• G06F11/3013

  where the computing system is an embedded system, i.e. a combination of hardware and software dedicated to perform a certain function in mobile devices, printers, automotive or aircraft systems (testing or monitoring of control systems or parts thereof G05B23/02) · CPC title

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F11/3089

  Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents (software debugging using additional hardware using a specific debug interface G06F11/3656; performance evaluation by tracing or monitoring G06F11/3466) · CPC title