Lane line detection method and related device

Patent metadata

Field	Value
Publication number	US-12154352-B2
Application number	US-202218064016-A
Country	US
Kind code	B2
Filing date	Dec 9, 2022
Priority date	Jun 12, 2020
Publication date	Nov 26, 2024
Grant date	Nov 26, 2024

Abstract

This disclosure discloses lane line detection methods and devices. In an implementation, features extracted by different layers of the neural network are fused to obtain a fused second feature map, so that the second feature map obtained through fusion processing has a plurality of layers of features. The fused second feature map has a related feature of a low-layer receptive field and a related feature of a high-layer receptive field. Afterwards, an output predicted lane line set is divided into groups, where each predicted lane line in each group has an optimal prediction interval.

First claim

What is claimed is:

1. A lane line detection method, comprising:
performing feature fusion on first feature maps that are output by different layers of a trained neural network, to obtain a second feature map, wherein input of the trained neural network is a to-be-detected image;
dividing the second feature map into a plurality of grids;
obtaining a first confidence level of each of n first grids in the plurality of grids, wherein the first confidence level indicates a first probability that a real lane line in the second feature map passes through the corresponding first grid, and the first probability exceeds a first preset threshold;
obtaining, by using a trained prediction head model, a first predicted lane line corresponding to each of the n first grids;
dividing n first predicted lane lines into m groups, wherein m≤n and n≥2, and wherein a first group in the m groups comprises q first predicted lane lines; and
integrating the q first predicted lane lines based on q first confidence levels corresponding to the q first predicted lane lines and q first grids corresponding to the q first predicted lane lines, to obtain a second predicted lane line, wherein the second predicted lane line is output as a detection result for a real lane line in the second feature map, and q≥2.

2. The method of claim 1 , wherein integrating the q first predicted lane lines comprises:
dividing a region occupied by the q first predicted lane lines in the second feature map into a plurality of sub-regions, wherein each sub-region comprises a central point location of at least one of q first grids, wherein a first sub-region in the plurality of sub-regions comprises central point locations of at least two first grids, and wherein a second sub-region in the plurality of sub-regions comprises a central point location of one first grid;
selecting a first grid from the at least two first grids as a second grid based on first confidence levels corresponding to the at least two first grids, and using, as a first part of the second predicted lane line, a part that is of a first predicted lane line corresponding to the second grid and that is located in the first sub-region; or performing a weighting operation on parts that are of first predicted lane lines corresponding to the at least two first grids and that are located in the first sub-region, and using an obtained operation result as the first part;
using, as a second part of the second predicted lane line, a part that is of a first predicted lane line corresponding to the first grid comprised in the second sub-region and that is located in the second sub-region; and
integrating the first part with the second part to obtain the second predicted lane line.

3. The method of claim 2 , wherein selecting the first grid from the at least two first grids as the second grid comprises:
selecting, from the at least two first grids based on the first confidence levels corresponding to the at least two first grids, a first grid that has a first confidence level of a maximum value as the second grid.

4. The method of claim 1 , wherein the first group in the m groups comprises one first predicted lane line, and wherein the method further comprises:
outputting the first predicted lane line in the first group as a detection result for a real lane line in the second feature map.

5. The method of claim 1 , wherein performing feature fusion on first feature maps to obtain the second feature map comprises:
constructing search space that is formed by combining code of a plurality of neural networks and code of a plurality of feature fusion models, and the plurality of feature fusion models are constructed of a plurality of preset rules;
performing sampling in the search space to obtain a first sampling point, wherein the first sampling point is a first code combination formed by code of a first neural network and code of a first feature fusion model;
generating a first lane line detection network based on the first code combination, and training the first lane line detection network by using a training set;
performing performance evaluation based on the trained first lane line detection network to obtain performance of the trained first lane line detection network;
in response to determining that a quantity of sampling times reaches a preset value and performance of the trained first lane line detection network satisfies a preset condition, extracting a feature from a to-be-detected image by using the first neural network in the trained first lane line detection network, to obtain first feature maps output by different layers of the first neural network; and
using the first feature maps as input of the first feature fusion model in the trained first lane line detection network, to output the second feature map.

6. The method of claim 5 , wherein the plurality of preset rules comprise at least one of the following:
separately operating at least two high-resolution first feature maps by using at least one first convolution kernel to obtain at least two third feature maps;
processing resolution of the at least two third feature maps to obtain at least two fourth feature maps having same resolution as a low-resolution first feature map; and
fusing the at least two fourth feature maps with the low-resolution first feature map in a preset combination manner to obtain at least one second feature map.

7. The method of claim 6 , wherein the processing resolution of the at least two third feature maps comprises at least one of the following manners:
performing a downsampling operation, a pooling operation, or a convolution operation on the at least two third feature maps.

8. The method of claim 5 , wherein the performance of the trained first lane line detection network satisfies the preset condition if the performance of the trained first lane line detection network is optimal in performance of lane line detection networks corresponding to all sampling points.

9. The method of claim 1 , wherein dividing the second feature map into a plurality of grids and obtaining a first confidence level of a first grid in the plurality of grids comprises:
dividing the second feature map into a plurality of grids and obtaining an initial confidence level of the first grid in the plurality of grids, wherein the initial confidence level indicates an initial probability that a real lane line in the second feature map passes through the first grid, and the initial probability exceeds an initial preset threshold; and
adjusting the initial confidence level in a preset manner to obtain a first confidence level of the first grid.

10. The method of claim 9 , wherein adjusting the initial confidence level in the preset manner to obtain the first confidence level of the first grid comprises:
establishing a statistical model based on distribution of central point locations of grids in which remote ends of real lane lines in images in a training set are located, wherein the statistical model has at least one parameter;
estimating the at least one parameter based on a statistical result for the central point locations of the grids in which the remote ends of the real lane lines in the images in the training set are located, to obtain an estimated value of the at least one parameter;
adjusting the estimated value of the at least one parameter in a training process to obtain a determinate value of the at least one parameter;
calculating a central point location of the first grid by using the statistical model for which the determinate value of the at least one parameter is determined, to obtain a weight value of the central point location of the first grid; and
performing an operation on the initial confidence level of t

Assignees

• Huawei Tech Co Ltd

Inventors

• Cai Xinyue
• Xu Hang
• Zhang Wei
• Yang Zhen
• Li Zhenguo

Classifications

• G06T2207/30256

  Lane; Road marking · CPC title

• G06T5/20

  using local operators · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/803

  of input or preprocessed data · CPC title

• G06V10/771

  Feature selection, e.g. selecting representative features from a multi-dimensional feature space · CPC title