Systems and methods to test an autonomous vehicle

Patent metadata

Field	Value
Publication number	US-11257309-B2
Application number	US-201916585482-A
Country	US
Kind code	B2
Filing date	Sep 27, 2019
Priority date	Aug 21, 2017
Publication date	Feb 22, 2022
Grant date	Feb 22, 2022

Abstract

The present disclosure provides systems and methods to test an autonomous vehicle. In particular, the systems and methods of the present disclosure can receive, from one or more test nodes of a preconfigured test track, log data indicating positions of elements of the test track over a period of time. Log data indicating parameters of an autonomous vehicle over the period of time can be received from the autonomous vehicle. The log data indicating the positions of the elements of the test track over the period of time can be compared with the log data indicating the parameters of the autonomous vehicle over the period of time to determine a performance metric of the autonomous vehicle on the test track over the period of time.

First claim

What is claimed is:

1. A computer-implemented method for autonomous vehicle testing, the method comprising:
receiving, by a computing system comprising one or more computing devices and from one or more test nodes of a preconfigured test track, log data indicating positions of elements of the test track over a period of time;
receiving, by the computing system and from an autonomous vehicle, log data indicating parameters of the autonomous vehicle over the period of time; and
comparing, by the computing system, the log data indicating the positions of the elements of the test track over the period of time with the log data indicating the parameters of the autonomous vehicle over the period of time to determine a performance metric of the autonomous vehicle on the test track over the period of time.

2. The computer-implemented method of claim 1 , wherein receiving the log data indicating the positions of the elements of the test track comprises receiving data indicating the position or status of one or more actuators associated with the elements.

3. The computer-implemented method of claim 1 , wherein receiving the log data indicating the positions of the elements of the test track comprises receiving data from one or more global positioning system (GPS) receivers attached to the elements.

4. The computer-implemented method of claim 1 , wherein receiving the log data indicating the parameters of the autonomous vehicle comprises receiving data from a global positioning system (GPS) receiver of the autonomous vehicle.

5. The computer-implemented method of claim 1 , wherein receiving the log data indicating the parameters of the autonomous vehicle comprises receiving data indicating one or more of a rate of acceleration of the autonomous vehicle, a rate of deceleration of the autonomous vehicle, a speed of the autonomous vehicle, or a heading of the autonomous vehicle.

6. The computer-implemented method of claim 1 , further comprising comparing the performance metric of the autonomous vehicle on the test track over the period of time with a performance metric of the autonomous vehicle on the test track over a previous period of time.

7. The computer-implemented method of claim 6 , further comprising, prior to the period of time and subsequent to the previous period of time, modifying, by the computing system, software configured to control operations of the autonomous vehicle.

8. The computer-implemented method of claim 1 , further comprising receiving, by the computing system from a user device, feedback describing performance of the autonomous vehicle over the period of time, wherein the performance metric is determined based at least in part on the feedback.

9. The computer-implemented method of claim 1 , wherein the comparing comprises determining a position of the autonomous vehicle relative to at least one of a simulated vehicle on the test track, a simulated pedestrian on the test track, or a simulated cyclist on the test track.

10. A computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
comparing log data indicating positions of elements of a test track over a first period of time with log data indicating parameters of an autonomous vehicle over the first period of time to determine a performance metric of the autonomous vehicle on the test track over the first period of time;
comparing log data indicating positions of the elements of the test track over a second period of time with log data indicating parameters of the autonomous vehicle over the second period of time to determine a performance metric of the autonomous vehicle on the test track over the second period of time; and
comparing the performance metric of the autonomous vehicle on the test track over the first period of time with the performance metric of the autonomous vehicle on the test track over the second period of time to determine an effect of modifying software configured to control operations of the autonomous vehicle.

11. The computer-implemented method of claim 1 , wherein the performance metric is associated with a change in motion during a transition from one state to another of the autonomous vehicle.

12. The computing system of claim 10 , wherein the operations further comprise:
receiving, from a user device, feedback describing performance of the autonomous vehicle over the first period of time and the second period of time; and
determining the effect of modifying the software configured to control operations of the autonomous vehicle based at least in part on the feedback.

13. The computing system of claim 10 , wherein:
the log data indicating the positions of the elements of the test track over the first period of time indicates a position of at least one of a simulated vehicle on the test track, a simulated pedestrian on the test track, or a simulated cyclist on the test track; and
the log data indicating the positions of the elements of the test track over the second period of time indicates a position of the at least one of the simulated vehicle on the test track, the simulated pedestrian on the test track, or the simulated cyclist on the test track.

14. The computing system of claim 10 , wherein the parameters of the autonomous vehicle comprise one or more of a rate of acceleration of the autonomous vehicle, a rate of deceleration of the autonomous vehicle, a speed of the autonomous vehicle, or a heading of the autonomous vehicle.

15. The computing system of claim 10 , wherein the performance metric is associated with a change in motion during a transition from one state to another of the autonomous vehicle.

16. One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a computing system to perform operations, the operations comprising:
comparing log data indicating positions of elements of a test track over a first period of time with log data indicating positions of the elements of the test track over a second period of time;
comparing log data indicating parameters of an autonomous vehicle over the first period of time with log data indicating the parameters of the autonomous vehicle over the second period of time; and
determining an effect of modifying, between the first period of time and the second period of time, software configured to control operations of the autonomous vehicle.

17. The one or more non-transitory computer-readable media of claim 16 , wherein the elements of the test track comprise at least one of a simulated vehicle, a simulated pedestrian, or a simulated cyclist.

18. The one or more non-transitory computer-readable media of claim 16 , wherein determining the effect of modifying the software comprises determining at least one of a rate of change of a speed of the autonomous vehicle or a heading of the autonomous vehicle in transition from one state to another of the autonomous vehicle.

19. The one or more non-transitory computer-readable media of claim 16 , wherein the parameters of the autonomous vehicle comprise one or more of a rate of acceleration of the autonomous vehicle, a rate of deceleration of the autonomous vehicle, a speed of the autonomous vehicle, or a heading of the autonomous vehicle.

20. The one or more non-transitory computer-readable media of claim 16 , wherein determining the effec

Assignees

• Uatc Llc

Inventors

• Watson Adam
• Mcintosh Chad David
• Bembic Anthony
• Leja Joseph Richard

Classifications

• G01M17/007 Primary

  Wheeled or endless-tracked vehicles (G01M17/08 takes precedence) · CPC title

• G01M17/0078

  Shock-testing of vehicles · CPC title

• G07C5/0841 Primary

  Registering performance data (recording measured values G01D; information storage G11B) · CPC title

• G07C5/08

  Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title