Automatic lane marking extraction and classification from lidar scans

Patent metadata

Field	Value
Publication number	US-2025308261-A1
Application number	US-202519239102-A
Country	US
Kind code	A1
Filing date	Jun 16, 2025
Priority date	Jun 1, 2022
Publication date	Oct 2, 2025
Grant date	—

Abstract

Provided are methods, systems, and computer program products for generating an output map indicating a likelihood of individual elements of an image as corresponding to particular road elements, such as lane dividers, road dividers, and road boundaries. An example method may include applying a machine learning architecture to the image, which architecture includes a convolutional neural network and a sub-network capturing global context from feature maps generated by the convolutional neural network.

First claim

What is claimed is:

1 . A computer-implemented method implemented by at least one processor, the computer-implemented method comprising:
receiving, by the at least one processor, an image representing data from a liDAR scan of a vehicle environment; convoluting, by the at least one processor, the image to generate a first feature map; converting, by the at least one processor, the first feature map into a first feature vector; applying, by the at least one processor, one or more additional convolutions to the first feature map to generate a second feature map; converting, by the at least one processor, the second feature map into a second feature vector; passing, by the at least one processor, an input representing at least the first feature vector and the second feature vector through a neural network to produce a third feature vector; deconvolving a combination of the third feature vector and the second feature map; and generating, by the at least one processor, an output map based on at least in part on the deconvolving of the combination of the third feature vector and the second feature map, wherein the output map includes a plurality of pixels and indicates a likelihood of each pixel corresponding to a road element.

2 . The computer-implemented method of claim 1 , wherein the image is a two-dimensional pseudo-image generated from a three-dimensional liDAR point cloud image.

3 . The computer-implemented method of claim 2 , wherein the three-dimensional liDAR point cloud image is generated from an aggregate three-dimensional liDAR point cloud image representing multiple liDAR scans of the vehicle environment.

4 . The computer-implemented method of claim 1 , wherein generating the output map comprises generating a segmentation map that associates a classification value to each pixel of the plurality of pixels, the classification value indicating the likelihood that the pixel corresponds to the road element.

5 . The computer-implemented method of claim 4 , wherein the road element is one of a plurality of road elements, and wherein the output map associates multiple classification values to each pixel, each of the multiple classification values corresponding to a respective road element of the plurality of road elements.

6 . The computer-implemented method of claim 5 , wherein the plurality of road elements comprises at least one of a lane divider indicating a division between lanes of traffic in a same direction, a road divider indicating a division between lanes of traffic in different directions, a road boundary element indicating a boundary of a roadway, stop lines, pedestrian markings, bike markings, and chevron markings.

7 . The computer-implemented method of claim 1 , wherein applying one or more additional convolutions to the first feature map to generate the second feature map comprises applying a first additional convolution to the first feature map to generate an intermediary feature map and applying a second additional convolution to the intermediary feature map to generate the second feature map.

8 . The computer-implemented method of claim 1 , wherein converting the first feature map into the first feature vector comprises reducing a dimensionality of the first feature map using a pooling operation.

9 . The computer-implemented method of claim 1 , wherein the input representing at least the first feature vector and the second feature vector is a concatenation of at least the first feature vector and the second feature vector.

10 . The computer-implemented method of claim 1 , further comprising generating one or more polylines from the output map, each of the one or more polylines identifying a road element in the vehicle environment.

11 . A system comprising:
one or more non-transitory data stores including computer-executable instructions; and one or more hardware processors configured to execute the computer-executable instructions to:
receive an image representing data from a liDAR scan of a vehicle environment;
convolute the image to generate a first feature map;
convert the first feature map into a first feature vector;
apply one or more additional convolutions to the first feature map to generate a second feature map;
convert the second feature map into a second feature vector;
pass an input representing at least the first feature vector and the second feature vector through a neural network to produce a third feature vector;
deconvolve a combination of the third feature vector and the second feature map; and
generate an output map based on at least in part on the deconvolution of the combination of the third feature vector and the second feature map, wherein the output map includes a plurality of pixels and indicates a likelihood of each pixel corresponding to a road element.

12 . The system of claim 11 , wherein the computer-executable instructions, when executed by the one or more hardware processors, further cause the system to generate one or more polylines from the output map, each of the one or more polylines identifying a road element in the vehicle environment.

13 . The system of claim 12 , wherein to generate the one or more polylines, the computer-executable instructions, when executed by the one or more hardware processors, further cause the system to skeletonize the output map.

14 . The system of claim 12 , wherein to generate the one or more polylines, the computer-executable instructions, when executed by the one or more hardware processors, further cause the system to:
convert one or more adjacent pixels in the output map with a classification value satisfying a threshold into an undirected graph; and identify a longest path within the undirected graph as a polyline of the one or more polylines.

15 . The system of claim 11 , wherein the image is a two-dimensional pseudo-image generated from a three-dimensional liDAR point cloud image.

16 . A computer-implemented method comprising:
receiving, by at least one processor, data corresponding to an image representing a liDAR scan of a vehicle environment; determining, by the at least one processor and using a machine learning model, a road element classification for a plurality of pixels of the image, wherein the classification for a particular pixel of the plurality of pixels indicates a traffic direction of a traffic lane associated with the particular pixel of the image; and generating, by the at least one processor, a plurality of polylines based on the classification for the plurality of pixels, wherein at least two polylines of the plurality of polylines indicate a boundary between at least two traffic lanes for traffic traveling in different directions.

17 . The computer-implemented method of claim 16 , wherein determining the road element classification for the plurality of pixels of the image representing data from the lidar scan comprises generating an output map, the output map comprising a plurality of pixels and indicates a likelihood of each pixel corresponding to a road element of a plurality or road elements.

18 . The computer-implemented method of claim 17 , wherein generating the plurality of polylines comprises skeletonizing the output map.

19 . The computer-implemented method of claim 17 , wherein generating the plurality of polylines comprises:
converting one or more adjacent pixels in the output map with a classification value satisfying a threshold into an undirected graph; and identifying a longest path within the undirected graph as a polyline of the one or more polylin

Assignees

• Motional Ad Llc

Inventors

• Sharma Dhananjai
• Liong Venice Erin Baylon
• Widjaja Sergi Adipraja
• Capellier Edouard Francois Marc

Classifications

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G06V10/82

  using neural networks · CPC title

• G06T7/10

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title