Autonomous vehicle lane boundary detection systems and methods

Patent metadata

Field	Value
Publication number	US-11682196-B2
Application number	US-202117359031-A
Country	US
Kind code	B2
Filing date	Jun 25, 2021
Priority date	Nov 15, 2017
Publication date	Jun 20, 2023
Grant date	Jun 20, 2023

Abstract

Systems and methods for facilitating communication with autonomous vehicles are provided. In one example embodiment, a computing system can obtain a first type of sensor data (e.g., camera image data) associated with a surrounding environment of an autonomous vehicle and/or a second type of sensor data (e.g., LIDAR data) associated with the surrounding environment of the autonomous vehicle. The computing system can generate overhead image data indicative of at least a portion of the surrounding environment of the autonomous vehicle based at least in part on the first and/or second types of sensor data. The computing system can determine one or more lane boundaries within the surrounding environment of the autonomous vehicle based at least in part on the overhead image data indicative of at least the portion of the surrounding environment of the autonomous vehicle and a machine-learned lane boundary detection model.

First claim

What is claimed is:

1. A vehicle computing system, comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that store instructions that when executed by the one or more processors cause the vehicle computing system to perform operations comprising:
obtaining a plurality of types of sensor data associated with a surrounding environment;
determining, using one or more machine-learned models, a ground height estimation of at least a portion of the surrounding environment based at least in part on a first type of sensor data of the plurality of types of sensor data;
transforming, using the ground height estimation, a second type of sensor data of the plurality of types of sensor data; and
determining one or more lane boundaries within the surrounding environment based at least in part on the transformed second type of sensor data.

2. The vehicle computing system of claim 1 , wherein the first type of sensor data comprises LIDAR data associated with the surrounding environment.

3. The vehicle computing system of claim 1 , wherein the second type of sensor data comprise image data associated with the surrounding environment.

4. The vehicle computing system of claim 3 , wherein determining, using the one or more machine-learned models, the ground height estimation comprises:
inputting the first type of sensor data into a first model of the one or more machine-learned models; and
obtaining, from the first model, the ground height estimation.

5. The vehicle computing system of claim 1 , wherein the second type of sensor data is transfored to a bird's-eye-view.

6. The vehicle computing system of claim 5 , wherein determining the one or more lane boundaries within the surrounding environment comprises:
inputting the transformed second type of sensor data; and
obtaining, from the second machine-learned model, an output indicative of the one or more lane boundaries.

7. The vehicle computing system of claim 1 , wherein the operations further comprise:
perceiving an object within the surrounding environment based at least in part on the one or more lane boundaries.

8. The vehicle computing system of claim 1 , wherein the operations further comprise:
predicting a motion trajectory of an object within the surrounding environment based at least in part on the one or more lane boundaries.

9. The vehicle computing system of claim 1 , wherein the vehicle computing system is onboard an autonomous vehicle, and wherein the operations further comprise:
determining a motion of the autonomous vehicle based at least in part on the one or more lane boundaries.

10. The vehicle computing system of claim 9 , wherein the autonomous vehicle is an autonomous truck.

11. An autonomous vehicle comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that store instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
obtaining a plurality of types of sensor data associated with a surrounding environment;
determining, using one or more machine-learned models, a ground height estimation of at least a portion of the surrounding environment based at least in part on a first type of sensor data of the plurality of types of sensor data;
transforming, using the ground height estimation, a second type of sensor data of the plurality of types of sensor data;
determining one or more lane boundaries within the surrounding environment based at least in part on the transformed second type of sensor data; and
initiating a performance of one or more vehicle actions by the autonomous vehicle based at least in part on the one or more lane boundaries.

12. The autonomous vehicle of claim 11 , wherein initiating the performance of the one or more vehicle actions by the autonomous vehicle based at least in part on the one or more lane boundaries comprises:
perceiving an object within the surrounding environment;
predicting a motion trajectory of the object based at least in part on the one or more lane boundaries; and
determining a motion of the autonomous vehicle based at least in part on the motion trajectory of the object.

13. The autonomous vehicle of claim 11 , wherein initiating the performance of the one or more vehicle actions by the autonomous vehicle based at least in part on the one or more lane boundaries comprises:
determining a location of the autonomous vehicle based at least in part on the one or more lane boundaries.

14. The autonomous vehicle of claim 11 , wherein determining, using the one or more machine-learned models, the ground height estimation comprises:
inputting the first type of sensor data into a first model of the one or more machine-learned models; and
obtaining, from the first model, the ground height estimation.

15. The autonomous vehicle of claim 11 , wherein the second type of sensor data is transformed to a bird's-eye-view.

16. The autonomous vehicle of claim 15 , wherein determining the one or more lane boundaries within the surrounding environment comprises:
inputting the transformed second type of sensor data into a second machine-learned model; and
obtaining, from the second machine-learned model, an output indicative of the one or more lane boundaries.

17. The autonomous vehicle of claim 11 , wherein the autonomous vehicle comprises a plurality of sensors, the plurality of sensors being different types of sensors, and wherein obtaining the plurality of types of sensor data associated with the surrounding environment comprises:
obtaining, through at least a portion of the plurality of sensors, the plurality of types of sensor data associated with the surrounding environment.

18. The autonomous vehicle of claim 11 , wherein the surrounding environment comprises a travel lane on a highway.

19. The autonomous vehicle of claim 18 , wherein the one or more lane boundaries are associated with the travel lane located on the highway.

20. A computer-implemented method comprising:
obtaining a plurality of types of sensor data associated with a surrounding environment;
determining, using one or more machine-learned models, a ground height estimation of at least a portion of the surrounding environment based at least in part on a first type of sensor data of the plurality of types of sensor data;
transforming, using the grond height estimation, a second type of sensor data of the plurality of types of sensor data; and
determining one or more lane boundaries within the surrounding environment based at least in part on the transformed second type of sensor data.

Assignees

• Uatc Llc

Inventors

• Bai Min
• Mattyus Gellert Sandor
• Homayounfar Namdar
• Wang Shenlong
• Lakshmikanth Shrindihi Kowshika
• Urtasun Raquel

Classifications

• G05D1/0221

  involving a learning process · CPC title

• G05D1/0231

  using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16) · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G01S17/931

  of land vehicles · CPC title