Lane line detection method, vehicle-mounted device and storage medium

Patent metadata

Field	Value
Publication number	US-12430929-B2
Application number	US-202318125680-A
Country	US
Kind code	B2
Filing date	Mar 23, 2023
Priority date	Dec 20, 2022
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

A lane line detection method applied to a vehicle-mounted device is provided. In the method, the vehicle-mounted device converts a foreground image into a bird's-eye view (BEV) image, sets a peak of a horizontal histogram of the BEV image as a starting point for moving the sliding window, and generate a main lane line by fitting multiple non-zero pixels of the horizontal sliding window. The vehicle-mounted device calculates a confidence coefficient of each horizontal sliding window; and sets a previous one before the preset number of continuous horizontal sliding windows as a previous horizontal sliding window and determines an end point of the main lane line according to the previous horizontal sliding window when the confidence coefficients of preset number of continuous horizontal sliding windows are less than a predetermined threshold. The method improves accuracy of identifying lane lines.

First claim

What is claimed is:

1. A lane line detection method using a vehicle-mounted device, the method comprising:
obtaining a foreground image of a vehicle;
converting the foreground image into a bird's eye view (BEV) image;
establishing a horizontal histogram image corresponding to the BEV image;
setting a peak of the horizontal histogram as a starting point while moving a horizontal sliding window, and generating a main lane line by fitting a plurality of non-zero pixels of the horizontal sliding window;
calculating a confidence coefficient of each horizontal sliding window;
in response that a plurality of confidence coefficients of a preset number of continuous horizontal sliding windows are less than a predetermined threshold, setting a sliding window that is previous before the preset number of continuous horizontal sliding windows as a previous horizontal sliding window, and determining an end of the main lane line according to the previous horizontal sliding window; and
establishing a vertical histogram according to the BEV image, and setting a peak of the vertical histogram as a starting point while moving a vertical sliding window;
generating a target curve by fitting non-zero pixels of the vertical sliding window; and
determining that the target curve is a stop line in response that an end point of the lane line being on the target curve.

2. The lane line detection method according to claim 1 , wherein converting the foreground image into the BEV image comprises:
obtaining a corrected image by performing a distortion correction on the foreground image, and
setting each non-zero pixel of the corrected image as a target point;
obtaining an inverse perspective transformation matrix by performing a coordinate transformation on each target point of the corrected image; and
converting the corrected image to the BEV image according to the inverse perspective transformation matrix.

3. The lane line detection method according to claim 1 , wherein setting the peak of the horizontal histogram as the starting point while moving the horizontal sliding window, and generating the main lane line by fitting the plurality of non-zero pixels of the horizontal sliding window comprises:
setting the peak of the horizontal histogram as the starting point for moving the horizontal sliding window;
calculating a first average value of the horizontal coordinates of all non-zero pixels in the previous horizontal sliding window of a current horizontal sliding window;
setting the first average value as a first horizontal coordinate average value;
determining a horizontal center of the current horizontal sliding window according to the first horizontal coordinate average value;
determining a position of the current horizontal sliding window according to the determined horizontal center; and
generating the main lane line by fitting all the non-zero pixels of the current horizontal sliding window and all horizontal sliding windows before the current horizontal sliding window.

4. The lane line detection method according to claim 1 , wherein calculating the confidence coefficient of each horizontal sliding window comprises:
inputting the BEV image comprising the horizontal sliding window into a predetermined deep learning neural network model;
calculating a similarity between a pixel feature of each pixel of each horizontal sliding window and a sample feature; and
determining the confidence coefficient of each horizontal sliding window based on the similarity, the similarity being proportional to the confidence coefficient.

5. The lane line detection method according to claim 1 , wherein setting the peak of the vertical histogram as the starting point while moving the vertical sliding window and generating the target curve by fitting the plurality of non-zero pixels of the vertical sliding window comprises:
setting a peak of the vertical sliding window as the starting point while moving the vertical sliding window;
calculating a second average value of the vertical coordinates of all non-zero pixels in the previous vertical sliding window of a current vertical sliding window;
setting the second average value as a second vertical coordinate average value;
determining a vertical center of the current vertical sliding window according to the second vertical coordinate average value;
determining a position of the current vertical sliding window according to the determined vertical center; and
generating the target curve by fitting all the non-zero pixels of the current vertical sliding window and all vertical sliding windows before the current vertical sliding window.

6. The lane line detection method according to claim 1 , wherein determining that the target curve is the stop line in response that the end point of the lane line being on the target curve comprises:
calculating at least one matching degree between a plurality of vertical sliding windows corresponding to the target curve and the previous horizontal sliding window;
determining the end of the main lane line is located on the target curve in response that any one of the at least one matching degree exceeds a preset matching degree; and
determining the position of the stop line according to the position of the target curve and the position of the end of the main lane line.

7. The lane line detection method according to claim 6 , wherein after determining the target curve is the stop line, the method further comprises:
filtering out the preset number of continues horizontal sliding windows.

8. A vehicle-mounted device comprising:
a storage device;
at least one processor; and
the storage device storing one or more programs, which when executed by the at least one processor, cause the at least one processor to:
obtain a foreground image of a vehicle;
convert the foreground image into a bird's eye view (BEV) image;
establish a horizontal histogram image corresponding to the BEV image;
set a peak of the horizontal histogram as a starting point while moving a horizontal sliding window, and generate a main lane line by fitting a plurality of non-zero pixels of the horizontal sliding window;
calculate a confidence coefficient of each horizontal sliding window;
in response that a plurality of confidence coefficients of a preset number of continuous horizontal sliding windows are less than a predetermined threshold, set a sliding window that is previous before the preset number of continuous horizontal sliding windows as a previous horizontal sliding window, and determine an end of the main lane line according to the previous horizontal sliding window; and
establish a vertical histogram according to the BEV image, and set a peak of the vertical histogram as a starting point while moving a vertical sliding window;
generate a target curve by fitting non-zero pixels of the vertical sliding window; and
determine that the target curve is a stop line in response that an end point of the lane line being on the target curve.

9. The vehicle-mounted device according to claim 8 , wherein the at least one processor converts the foreground image into the BEV image by:
obtaining a corrected image by performing a distortion correction on the foreground image;
setting each non-zero pixel of the corrected image as a target point;
obtaining an inverse perspective transformation matrix by performing a coordinate transformation on each target point of the corrected image; and
converting the corrected image to the BEV image according to the inverse perspective transformation matrix.

10. The vehicle-mounted device according to claim 8 , wherein the at least one processor sets the peak of the horizontal histogram as the starting point while moving the ho

Assignees

• Hon Hai Prec Ind Co Ltd

Inventors

• Lee Chieh
• Kuo Chin-Pin

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06V10/28

  Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns · CPC title

• G06V10/24

  Aligning, centring, orientation detection or correction of the image · CPC title