Systems and methods for training a vehicle to autonomously drive a route

Patent metadata

Field	Value
Publication number	US-11294387-B2
Application number	US-201916442904-A
Country	US
Kind code	B2
Filing date	Jun 17, 2019
Priority date	Jun 17, 2019
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

A system for determining when a vehicle has learned a route is provided. The system includes at least one camera configured to capture image data of an external environment, a processor and a memory module storing one or more processor-readable instructions that, when executed by the processor, cause the system to: generate one or more predictions about objects in the external environment as the vehicle proceeds along the route based on the image data, determine a prediction error based on an accuracy of the one or more predictions, determine a confidence level for the route based on the prediction error, and determine that the vehicle has learned the route in response to the confidence level exceeding a confidence level threshold.

First claim

What is claimed is:

1. A system for determining when a vehicle has learned a route comprising:
at least one camera configured to capture image data of an external environment;
a processor and a memory module storing one or more processor-readable instructions that, when executed by the processor, cause the system to:
generate one or more predictions about objects in the external environment as the vehicle proceeds along the route based on the image data;
determine a prediction error based on an accuracy of the one or more predictions;
determine a confidence level for the route based on the prediction error; and
determine that the vehicle has learned the route in response to the confidence level exceeding a confidence level threshold.

2. The system of claim 1 , wherein the system is configured to reduce a frequency of capturing and recording prediction data based on the confidence level exceeding a second confidence level threshold.

3. The system of claim 1 , further comprising:
one or more position detection sensors configured to determine a position of the vehicle, and wherein
the one or more processor-readable instructions further cause the system to:
capture and record a first position of the vehicle as base position data;
capture and record a second position of the vehicle as prediction position data;
compare the base position data with the prediction position data;
determine the prediction error based on the comparison between the base position data and the prediction position data;
determine the confidence level for the route based on the comparison between the base position data and the prediction position data; and
operate the vehicle in an autonomous mode along the route based on the confidence level.

4. The system of claim 1 , wherein the one or more processor-readable instructions further cause the system to generate an alert when the confidence level for the route exceeds the confidence level threshold to inform a user that the system is prepared to autonomously operate the vehicle.

5. The system of claim 1 , wherein the one or more processor-readable instructions further cause the system to:
operate the vehicle in an autonomous mode along the route based on the confidence level exceeding the confidence level threshold.

6. The system of claim 1 , wherein the one or more processor-readable instructions further cause the system to:
capture and record image data of the route at a first time as base data;
capture and record image data of the route at a second time as prediction data; and
compare the base data and the prediction data, wherein the one or more predictions about objects in the external environment is generated based on the comparison of the base data and the prediction data.

7. The system of claim 6 , further comprising:
one or more of:
one or more lidar sensors;
one or more sonar sensors; and
one or more radar sensors, wherein the base data and the prediction data include one or more of lidar, sonar, and radar data.

8. The system of claim 6 further comprising:
network interface hardware, wherein
the one or more processor-readable instructions, when executed, further cause the system to:
access an external network; and
upload the base data as content to a library of external image data on the external network such that it can be accessed and used by other vehicles with access to the content of the library of external image data on the external network.

9. The system of claim 1 , wherein the system is configured to operate in a training mode and in an autonomous mode such that:
in the training mode a user controls at least the steering, propulsion, and braking systems of the vehicle, and
in the autonomous mode, the vehicle autonomously operates one or more of the steering, propulsion, and the braking systems of the vehicle.

10. The system of claim 9 , wherein the one or more processor-readable instructions further cause the system to automatically shift the vehicle to the training mode based on the confidence level dropping below the confidence level threshold.

11. The system of claim 8 , wherein the one or more processor-readable instructions, when executed, further cause the system to:
access the external network;
download content from the library of external image data;
compare the content to one or more of the base data and the prediction data; and
generate the prediction error based on the comparison between the content and the one or more of the base data and the prediction data.

12. The system of claim 8 , wherein the one or more processor-readable instructions, when executed, further cause the system:
download traffic information from the external network; and
make one or more recommendations for travelling the route based on traffic information.

13. A vehicle including a system for determining when the vehicle has learned a segment of a route comprising:
at least one camera configured to capture image data of an external environment;
a processor and a memory module storing one or more processor-readable instructions that, when executed by the processor, cause the system to:
generate one or more predictions about objects in the external environment based on the image data;
determine a prediction error based on an accuracy of the one or more predictions;
determine a confidence level for the segment based on the prediction error; and
determine that the vehicle has learned the segment of the route in response to the confidence level exceeding a confidence level threshold.

14. The vehicle of claim 13 , wherein the one or more processor-readable instructions further cause the system to:
capture and record image data of the route at a first time as base data;
capture and record image data of the route at a second time as prediction data; and
compare the base data and the prediction data, wherein the one or more predictions about objects in the external environment is generated based on the comparison of the base data and the prediction data.

15. The vehicle of claim 13 , wherein the one or more processor-readable instructions further cause the system to:
operate the vehicle in an autonomous mode along the segment based on the confidence level exceeding the confidence level threshold.

16. The vehicle of claim 13 , wherein the vehicle is configured to operate in a training mode and in an autonomous mode such that:
in the training mode a user controls at least the steering, propulsion, and braking systems of the vehicle, and
in the autonomous mode, the vehicle autonomously operates one or more of the steering, propulsion, and braking systems of the vehicle.

17. The vehicle of claim 16 , wherein the vehicle automatically shifts to the autonomous mode based on:
identifying a particular segment; and
the confidence level for the particular segment being above the confidence level threshold.

18. A method of autonomously operating a vehicle that includes a system including at least one camera configured to capture image data of an external environment along a route, the method comprising:
generating one or more predictions about objects in the external environment as the vehicle proceeds along the route based on the image data;
determining a prediction error based on an accuracy of the one or more predictions;
determining a confidence level for the route based on the prediction error; and
determining that the vehicle has learned the route in response to the confidence level exceeding a confidence level threshold.

19. The method of

Assignees

• Toyota Res Inst Inc

Inventors

• Northcutt Brandon D
• Gregg Christopher
• Carvalho Santana Jr Ewaldo Eder

Classifications

• G05B13/0265 Primary

  the criterion being a learning criterion · CPC title

• G05B17/02

  electric · CPC title

• G05D1/0221 Primary

  involving a learning process · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G05D2201/0213

  Physics · mapped topic