Systems and methods for autonomous vehicle performance evaluation

Patent metadata

Field	Value
Publication number	US-11953333-B2
Application number	US-201916700835-A
Country	US
Kind code	B2
Filing date	Dec 2, 2019
Priority date	Mar 6, 2019
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

Systems, methods, and non-transitory computer-readable media can receive transportation information associated with a transportation request, the transportation information comprising a pick up location and a drop off location. A first route associated with the transportation request and a non-autonomous vehicle can be determined. A second route associated with the transportation request and an autonomous vehicle can be determined based on an operating design domain (ODD) associated with one or more autonomous vehicles in a fleet of vehicles. At least one performance metric associated with the second route can be determined. The second route can be selected based at least in part on the at least one performance metric and a comparison of the first route and the second route. An autonomous vehicle from the fleet of vehicles can be assigned to the transportation request based on selection of the second route.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, by a computing system, transportation information associated with a transportation request, the transportation information comprising a pick up location and a drop off location;
determining, by the computing system, a first route associated with the transportation request and a non-autonomous vehicle;
determining, by the computing system, a second route associated with the transportation request and an autonomous vehicle based on an operating design domain (ODD) associated with one or more autonomous vehicles in a fleet of vehicles;
categorizing, by the computing system, disengagements in scenarios associated with the ODD into at least an adverse outcome category and a no adverse outcome category based on expected outcomes determined by simulation of the disengagements;
determining, by the computing system, at least one performance metric associated with the second route based on a weighted average of performance metric values associated with road segments of the second route, wherein the performance metric values are based on the expected outcomes of the disengagements in the adverse outcome category, wherein at least one of the road segments has a default performance metric value determined based on other road segments within a threshold distance of the at least one of the road segments, and wherein the performance metric values are weighted based on frequency by which the road segments were traversed;
selecting, by the computing system, the second route based at least in part on the at least one performance metric and a comparison of the first route and the second route;
assigning, by the computing system, an autonomous vehicle from the fleet of vehicles to the transportation request based on selection of the second route; and
dispatching, by the computing system, the autonomous vehicle to respond to the transportation request based on the assigning.

2. The computer-implemented method of claim 1 , wherein the ODD is defined based on at least one of environmental factors, map elements, or scenarios that the one or more autonomous vehicles are designed to handle.

3. The computer-implemented method of claim 1 , wherein the at least one performance metric includes a safety metric determined based on disengagement information.

4. The computer-implemented method of claim 1 , wherein the performance metric values are based on the expected outcomes of the disengagements in the adverse outcome category over a period of time, and wherein the default performance is determined based on the other road segments within the threshold distance of the at least one of the road segments and traversed within a threshold time of the period of time.

5. The computer-implemented method of claim 1 , wherein the determining the at least one performance metric comprises:
identifying, by the computing system, a set of disengagements in the adverse outcome category associated with each road segment; and
determining, by the computing system, a first performance metric based on disengagement information that includes the set of disengagements.

6. The computer-implemented method of claim 1 , further comprising:
determining, by the computing system, performance metric values for the at least one performance metric associated with road segments of the second route, based on a number of expected adverse events per number of miles traveled for each road segment of the second route.

7. The computer-implemented method of claim 1 , wherein the simulation of the disengagements is based on an assumption that external bodies continue behaviors indicated by sensor data captured prior to the disengagements.

8. The computer-implemented method of claim 1 , wherein the determining the at least one performance metric associated with the second route comprises:
determining, by the computing system, for each road segment of the second route, a performance metric based on unplanned disengagement information associated with the road segment.

9. The computer-implemented method of claim 1 , wherein the selecting the second route comprises:
determining, by the computing system, whether a route exists from the pick up location to the drop off location such that each road segment in the route satisfies a minimum performance metric threshold.

10. The computer-implemented method of claim 1 , wherein the selecting the second route comprises:
evaluating at least one of a potential time delay, a distance between a current location and the pickup location, a distance from the drop off location to a destination, or comfort level.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
receiving transportation information associated with a transportation request, the transportation information comprising a pick up location and a drop off location;
determining a first route associated with the transportation request and a non-autonomous vehicle;
determining a second route associated with the transportation request and an autonomous vehicle based on an operating design domain (ODD) associated with one or more autonomous vehicles in a fleet of vehicles;
categorizing disengagements in scenarios associated with the ODD into at least an adverse outcome category and a no adverse outcome category based on expected outcomes determined by simulation of the disengagements;
determining at least one performance metric associated with the second route based on a weighted average of performance metric values associated with road segments of the second route, wherein the performance metric values are based on the expected outcomes of the disengagements in the adverse outcome category, wherein at least one of the road segments has a default performance metric value determined based on other road segments within a threshold distance of the at least one of the road segments, and wherein the performance metric values are weighted based on frequency by which the road segments were traversed;
selecting the second route based at least in part on the at least one performance metric and a comparison of the first route and the second route;
assigning an autonomous vehicle from the fleet of vehicles to the transportation request based on selection of the second route; and
dispatching the autonomous vehicle to respond to the transportation request based on the assigning.

12. The system of claim 11 , wherein the ODD is defined based on at least one of environmental factors, map elements, or scenarios that the one or more autonomous vehicles are designed to handle.

13. The system of claim 11 , wherein the at least one performance metric includes a safety metric determined based on disengagement information.

14. The system of claim 11 , wherein the performance metric values are based on the expected outcomes of the disengagements in the adverse outcome category over a period of time, and wherein the default performance is determined based on the other road segments within the threshold distance of the at least one of the road segments and traversed within a threshold time of the period of time.

15. The system of claim 11 , wherein the determining the at least one performance metric comprises:
identifying a set of disengagements in the adverse outcome category associated with each road segment; and
determining a first performance metric based on disengagement information that includes the set of disengagements.

16. A non-transitory computer-readable storage medium including instruc

Assignees

• Lyft Inc

Inventors

• Chellapilla Kumar Hemachandra
• Danna Emilie Jeanne Anne
• Lu David Tse-Zhou
• Qureshi Sameer
• Weill Alexis

Classifications

• G01C21/3453 Primary

  Special cost functions, i.e. other than distance or default speed limit of road segments · CPC title

• G01C21/3438

  Rendezvous; Ride sharing · CPC title

• G01C21/3461

  Preferred or disfavoured areas, e.g. dangerous zones, toll or emission zones, intersections, manoeuvre types or segments such as motorways, toll roads or ferries · CPC title

• G06Q10/02 Primary

  Reservations, e.g. for tickets, services or events · CPC title

• G06Q50/30

  Physics · mapped topic