Autonomous vehicle lane boundary detection systems and methods

Patent metadata

Field	Value
Publication number	US-11972606-B2
Application number	US-202318313794-A
Country	US
Kind code	B2
Filing date	May 8, 2023
Priority date	Nov 15, 2017
Publication date	Apr 30, 2024
Grant date	Apr 30, 2024

Abstract

Systems and methods for facilitating communication with autonomous vehicles are provided. In one example embodiment, a computing system can obtain a first type of sensor data (e.g., camera image data) associated with a surrounding environment of an autonomous vehicle and/or a second type of sensor data (e.g., LIDAR data) associated with the surrounding environment of the autonomous vehicle. The computing system can generate overhead image data indicative of at least a portion of the surrounding environment of the autonomous vehicle based at least in part on the first and/or second types of sensor data. The computing system can determine one or more lane boundaries within the surrounding environment of the autonomous vehicle based at least in part on the overhead image data indicative of at least the portion of the surrounding environment of the autonomous vehicle and a machine-learned lane boundary detection model.

First claim

What is claimed is:

1. An autonomous vehicle control system for controlling an autonomous vehicle, the autonomous vehicle control system comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that store instructions that are executable by the one or more processors to cause the autonomous vehicle computing system to perform operations, the operations comprising:
obtaining, from a camera system of the autonomous vehicle, image data descriptive of a portion of a surrounding environment;
obtaining ground height data for the portion of the surrounding environment, wherein the ground height data was generated using a machine-learned model configured to process sensor data and output data indicative of ground height;
associating, using the ground height data, one or more portions of the image data with a lane of a travel way based on a transformation between a reference frame of the camera system and a reference frame of the ground height data; and
based on the association between the one or more portions of the image data and the lane, determining an object within the surrounding environment is within the lane.

2. The autonomous vehicle control system of claim 1 , the operations comprising:
based on determining the object is within the lane, determining a motion of the object.

3. The autonomous vehicle control system of claim 2 , wherein determining the motion comprises:
determining a current speed of the object or a past speed of the object.

4. The autonomous vehicle control system of claim 1 , wherein the transformation is based on a projection from the reference frame of the ground height data to the reference frame of the camera system.

5. The autonomous vehicle control system of claim 1 , wherein the ground height data is estimated ground height data generated based on LIDAR data.

6. The autonomous vehicle control system of claim 1 , the operations comprising:
obtaining map data descriptive of a location of the lane; and
generating perception data based on the map data.

7. The autonomous vehicle control system of claim 6 , the operations comprising:
obtaining sensor data comprising the image data and LIDAR data; and
processing the sensor data and the map data to generate the perception data.

8. One or more tangible, non-transitory, computer readable media that store instructions that are executable by one or more processors to cause an autonomous vehicle computing system to perform operations, the operations comprising:
obtaining, from a camera system of an autonomous vehicle, image data descriptive of a portion of a surrounding environment;
obtaining ground height data for the portion of the surrounding environment, wherein the ground height data was generated using a machine-learned model configured to process sensor data and output data indicative of ground height;
associating, using the ground height data, one or more portions of the image data with a lane of a travel way based on a transformation between a reference frame of the camera system and a reference frame of the ground height data; and
based on the association between the one or more portions of the image data and the lane, determining an object within the surrounding environment is within the lane.

9. The one or more computer readable media of claim 8 , the operations comprising:
based on determining the object is within the lane, determining a motion of the object.

10. The one or more computer readable media of claim 9 , wherein determining the motion comprises:
determining a current speed of the object or a past speed of the object.

11. The one or more computer readable media of claim 8 , wherein the transformation is based on a projection from the reference frame of the ground height data to the reference frame of the camera system.

12. The one or more computer readable media of claim 8 , wherein the ground height data is estimated ground height data generated based on LIDAR data.

13. The one or more computer readable media of claim 8 , the operations comprising:
obtaining map data descriptive of a location of the lane; and
generating perception data based on the map data.

14. The one or more computer readable media of claim 13 , the operations comprising:
obtaining sensor data comprising the image data and LIDAR data; and
processing the sensor data and the map data to generate the perception data.

15. A computer-implemented method comprising:
obtaining, from a camera system of an autonomous vehicle, image data descriptive of a portion of a surrounding environment;
obtaining ground height data for the portion of the surrounding environment, wherein the ground height data was generated using a machine-learned model configured to process sensor data and output data indicative of ground height;
associating, using the ground height data, one or more portions of the image data with a lane of a travel way based on a transformation between a reference frame of the camera system and a reference frame of the ground height data; and
based on the association between the one or more portions of the image data and the lane, determining an object within the surrounding environment is within the lane.

16. The computer-implemented method of claim 15 , comprising:
based on determining the object is within the lane, determining a motion of the object.

17. The computer-implemented method of claim 16 , wherein determining the motion comprises:
determining a current speed of the object or a past speed of the object.

18. The computer-implemented method of claim 15 , wherein the transformation is based on a projection from the reference frame of the ground height data to the reference frame of the camera system.

19. The computer-implemented method of claim 15 , wherein the ground height data is estimated ground height data generated based on LIDAR data.

20. The computer-implemented method of claim 15 , comprising:
obtaining sensor data comprising the image data and LIDAR data;
obtaining map data descriptive of a location of the lane; and
processing the sensor data and the map data to generate perception data.

Assignees

• Uatc Llc

Inventors

• Bai Min
• Mattyus Gellert Sandor
• Homayounfar Namdar
• Wang Shenlong
• Lakshmikanth Shrinidihi Kowshika
• Urtasun Raquel

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/82 Primary

  using neural networks · CPC title