Autonomous vehicle operation feature monitoring and evaluation of effectiveness

Patent metadata

Field	Value
Publication number	US-12140959-B2
Application number	US-202318149488-A
Country	US
Kind code	B2
Filing date	Jan 3, 2023
Priority date	May 20, 2014
Publication date	Nov 12, 2024
Grant date	Nov 12, 2024

Abstract

Methods and systems for monitoring use and determining risks associated with operation of a vehicle having one or more autonomous operation features are provided. According to certain aspects, operating data may be recorded during operation of the vehicle. This may include information regarding the vehicle, the vehicle environment, use of the autonomous operation features, and/or control decisions made by the features. The control decisions may include actions the feature would have taken to control the vehicle, but which were not taken because a vehicle operator was controlling the relevant aspect of vehicle operation at the time. The operating data may be recorded in a log, which may then be used to determine risk levels associated with vehicle operation based upon risk levels associated with the autonomous operation features. The risk levels may further be used to adjust an insurance policy associated with the vehicle.

First claim

What is claimed is:

1. A computer system for evaluating operation of an autonomous operation feature for controlling vehicle operation, comprising:
one or more processors; and
a non-transitory program memory coupled to the one or more processors and storing executable instructions that, when executed by the one or more processors, cause the computer system to:
receive indications of a plurality of vehicle collisions involving a plurality of vehicles having the autonomous operation feature;
for each vehicle collision of the plurality of vehicle collisions involving a respective vehicle of the plurality of vehicles:
receive sensor data from one or more sensors within the vehicle indicating (i) one or more environmental conditions in which the vehicle collision occurred, (ii) a person positioned within the vehicle to operate the vehicle at the time of the vehicle collision, and (iii) one or more capabilities or features of the autonomous operation feature of the vehicle;
determine one or more preferred control decisions the autonomous operation feature could have made to control the vehicle to reduce a risk of collision or mitigate an effect of the vehicle collision immediately before or during the vehicle collision based upon analysis of the sensor data using a trained machine learning program that has been previously trained to predict preferred control decisions under a plurality of operating conditions associated with corresponding sets of training sensor data;
receive control decision data indicating one or more actual control decisions the autonomous operation feature of the vehicle made to control the vehicle immediately before or during the vehicle collision; and
assign a degree of fault for the vehicle collision to the autonomous operation feature based upon an extent of consistency or inconsistency between

the one or more preferred control decisions and the one or more actual control decisions; and
determine a risk level for the autonomous operation feature based upon the respective degrees of fault for the plurality of vehicle collisions.

2. The computer system of claim 1 , wherein each of the one or more preferred control decisions and the one or more actual control decisions are virtually time-stamped for comparison of such controlled and actual control decisions based upon matching virtual time stamps.

3. The computer system of claim 1 , wherein the risk level is associated with one or more sets of parameters indicating configurations or settings of the autonomous operation feature.

4. The computer system of claim 1 , wherein the risk level is associated with a weighted average of a plurality of risk levels associated with operation of the autonomous operation feature under a plurality of sets of conditions comprising one or more of the following conditions: environmental conditions, road conditions, construction conditions, or traffic conditions.

5. The computer system of claim 1 , wherein the executable instructions that cause the computer system to determine the risk level for the autonomous operation feature cause the computer system to adjust an initial risk level determined based upon testing the autonomous operation feature in a test environment.

6. The computer system of claim 5 , wherein the test environment is a virtual test environment configured to present a plurality of sets of virtual environmental conditions to the autonomous operation feature in a plurality of virtual test scenarios.

7. The computer system of claim 1 , wherein the sensor data indicating (i) one or more environmental conditions in which the vehicle collision occurred or (ii) the person positioned within the vehicle to operate the vehicle at the time of the vehicle collision includes at least one of camera image data, radar unit data, or infrared data.

8. A tangible, non-transitory computer-readable medium storing executable instructions for evaluating operation of an autonomous operation feature for controlling vehicle operation that, when executed by at least one processor of a computer system, cause the computer system to:
receive indications of a plurality of vehicle collisions involving a plurality of vehicles having the autonomous operation features;
for each vehicle collision of the plurality of vehicle collisions involving a respective vehicle of the plurality of vehicles:
receive sensor data from one or more sensors within the vehicle indicating (i) one or more environmental conditions in which the vehicle collision occurred, (ii) a person positioned within the vehicle to operate the vehicle at the time of the vehicle collision, and (iii) one or more capabilities or features of the autonomous operation feature of the vehicle;
determine one or more preferred control decisions the autonomous operation feature could have made to control the vehicle to reduce a risk of collision or mitigate an effect of the vehicle collision immediately before or during the vehicle collision based upon analysis of the sensor data using a trained machine learning program that has been previously trained to predict preferred control decisions under a plurality of operating conditions associated with corresponding sets of training sensor data;
receive control decision data indicating one or more actual control decisions the autonomous operation feature of the vehicle made to control the vehicle immediately before or during the vehicle collision; and
assign a degree of fault for the vehicle collision to the autonomous operation feature based upon an extent of consistency or inconsistency between the one or more preferred control decisions and the one or more actual control decisions; and

determine a risk level for the autonomous operation feature based upon the respective degrees of fault for the plurality of vehicle collisions.

9. The tangible, non-transitory computer-readable medium of claim 8 , wherein each of the one or more preferred control decisions and the one or more actual control decisions are virtually time-stamped for comparison of such controlled and actual control decisions based upon matching virtual time stamps.

10. The tangible, non-transitory computer-readable medium of claim 8 , wherein the risk level is associated with a weighted average of a plurality of risk levels associated with operation of the autonomous operation feature under a plurality of sets of conditions comprising one or more of the following conditions: environmental conditions, road conditions, construction conditions, or traffic conditions.

11. The tangible, non-transitory computer-readable medium of claim 8 , wherein the executable instructions that cause the computer system to determine the risk level for the autonomous operation feature cause the computer system to adjust an initial risk level determined based upon testing the autonomous operation feature in a test environment.

12. The tangible, non-transitory computer-readable medium of claim 11 , wherein the test environment is a virtual test environment configured to present a plurality of sets of virtual environmental conditions to the autonomous operation feature in a plurality of virtual test scenarios.

13. The tangible, non-transitory computer-readable medium of claim 8 , wherein the sensor data indicating (i) one or more environmental conditions in which the vehicle collision occurred or (ii) the person positioned within the vehicle to operate the vehicle at the time of the vehicle collision includes at least one of camera image data, radar unit data, or infrared data.

14. A computer-implemented method of evaluating operation of an autonomous operation feature for controlling vehicle operation, the method c

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Konrardy Blake
• Christensen Scott T
• Hayward Gregory
• Farris Scott

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/08

  Learning methods · CPC title

• G05D1/81

  Handing over between on-board automatic and on-board manual control · CPC title

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title